Jul 15 04:56:38.476893 ip-10-0-130-29 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 15 04:56:38.476905 ip-10-0-130-29 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 15 04:56:38.476914 ip-10-0-130-29 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 15 04:56:38.477219 ip-10-0-130-29 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 15 04:56:48.483212 ip-10-0-130-29 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 15 04:56:48.483232 ip-10-0-130-29 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d6e970db291f4136bf13d40f91f39f72 --
Jul 15 04:58:41.717466 ip-10-0-130-29 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.132219    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136478    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136490    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136493    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136496    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136499    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136502    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136505    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:42.281075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136508    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:42.210407 ip-10-0-130-29 systemd[1]: Started Kubernetes Kubelet.
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136511    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136513    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136516    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136519    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136521    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136524    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136526    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136529    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136532    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136535    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136537    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136541    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136543    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136546    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136549    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136551    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136554    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136557    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136560    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:42.282075 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136563    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136565    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136568    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136570    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136573    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136575    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136578    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136580    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136583    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136586    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136589    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136592    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136595    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136597    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136600    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136602    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136608    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136612    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136615    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:42.283007 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136617    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136621    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136623    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136626    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136628    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136631    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136633    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136636    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136640    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136644    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136646    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136651    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136655    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136658    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136661    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136663    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136666    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136669    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136671    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:42.283737 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136674    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136676    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136679    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136682    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136684    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136687    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136690    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136692    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136695    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136697    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136700    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136703    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136705    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136708    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136712    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136714    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136717    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136719    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136722    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136724    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:42.284569 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136727    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136730    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136732    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136735    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136737    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136740    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136743    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136745    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136748    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136750    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136753    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136755    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136758    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.136761    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137604    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137613    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137618    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137622    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137626    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137631    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:42.285195 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137636    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137640    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137643    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137647    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137651    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137654    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137658    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137664    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137669    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137674    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137679    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137683    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137687    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137691    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137695    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137699    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137703    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137707    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137711    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:42.285936 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137714    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137718    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137723    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137727    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137732    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137735    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137738    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137740    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137743    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137746    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137750    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137754    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137758    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137763    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137767    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137770    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137776    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137780    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137783    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:42.286533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137787    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137790    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137794    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137798    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137802    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137806    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137810    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137814    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137818    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137822    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137826    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137830    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137834    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137841    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137846    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137850    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137855    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137859    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137863    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:42.287364 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137867    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137870    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137874    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137878    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137882    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137886    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137890    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137893    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137899    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137902    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137906    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137909    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137913    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137917    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137920    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137924    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137928    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137932    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137937    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137941    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:42.287942 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137944    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137946    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137950    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137953    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137958    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137961    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137965    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137970    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137973    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137977    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137980    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137984    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137988    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137992    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137996    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.137999    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138512    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138519    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138523    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138527    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:42.481812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138534    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138540    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138545    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138550    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138552    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138555    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138559    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138563    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138568    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138573    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138577    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138581    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138585    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138589    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138593    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138597    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138601    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138606    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138611    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:42.483390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138614    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138618    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138622    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138625    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138629    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138633    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138637    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138641    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138645    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138648    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138653    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138657    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138660    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138664    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138668    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138675    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138679    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138682    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138686    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138690    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:42.484220 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138693    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138697    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138701    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138705    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138709    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138712    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138716    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138720    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138723    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138727    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138733    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138738    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138742    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138748    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138753    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138757    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138760    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138763    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138767    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:42.485508 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138771    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138774    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138778    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138782    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138785    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138789    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138793    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138796    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138802    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138806    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138809    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138813    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138818    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138822    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138826    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138829    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138833    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138837    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138840    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138844    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:42.486115 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138848    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138852    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138855    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138859    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138863    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138867    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138870    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138874    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138878    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138882    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138885    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138889    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138893    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138896    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138900    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138904    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.138907    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.138991    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139000    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139009    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139017    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 04:58:42.486947 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139026    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139031    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139037    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139043    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139049    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139053    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139058    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139063    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139067    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139072    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139076    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139080    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139085    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139089    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139095    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139099    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139103    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139108    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139112    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139117    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139122    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139126    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139131    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139135    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 04:58:42.487813 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139139    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139144    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139148    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139153    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139159    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139162    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139165    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139171    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139175    2581 flags.go:64] FLAG: --enable-server="true"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139180    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139186    2581 flags.go:64] FLAG: --event-burst="100"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139190    2581 flags.go:64] FLAG: --event-qps="50"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139194    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139197    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139215    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139220    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139223    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139228    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139231    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139238    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139241    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139244    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139248    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139252    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139255    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 04:58:42.488571 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139259    2581 flags.go:64] FLAG: --feature-gates=""
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139264    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139267    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139271    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139275    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139278    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139282    2581 flags.go:64] FLAG: --help="false"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139286    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139289    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139292    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139297    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139300    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139303    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139307    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139312    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139314    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139318    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139323    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139326    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139330    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139334    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139337    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139340    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139343    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 04:58:42.490757 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139347    2581 flags.go:64] FLAG: --lock-file=""
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139349    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139356    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139359    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139364    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139368    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139370    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139389    2581 flags.go:64] FLAG: --logging-format="text"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139393    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139398    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139402    2581 flags.go:64] FLAG: --manifest-url=""
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139405    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139410    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139413    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139417    2581 flags.go:64] FLAG: --max-pods="110"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139421    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139424    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139427    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139431    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139435    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139438    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139442    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139451    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139455    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139458    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 04:58:42.491753 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139461    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139466    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139471    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139474    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139479    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139483    2581 flags.go:64] FLAG: --port="10250"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139486    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139491    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-016c6b4b8a556743e"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139494    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139498    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139503    2581 flags.go:64] FLAG: --register-node="true"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139505    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139510    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139514    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139517    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139520    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139525    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139528    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139532    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139536    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139539    2581 flags.go:64] FLAG: --runonce="false"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139542    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139547    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139550    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139554    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 04:58:42.685672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139557    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139560    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139563    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139566    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139570    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139572    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139576    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139579    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139582    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139587    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139590    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139596    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139600    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139603    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139607    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139611    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139614    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139619    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139624    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139627    2581 flags.go:64] FLAG: --v="2"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139632    2581 flags.go:64] FLAG: --version="false"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139637    2581 flags.go:64] FLAG: --vmodule=""
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139640    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.139644    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 04:58:42.686499 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139768    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139773    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139779    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139782    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139785    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139789    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139792    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139794    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139798    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139801    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139804    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139807    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139813    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139815    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139818    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139822    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139825    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139827    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139831    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139835    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:42.687812 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139838    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139840    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139845    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139847    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139849    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139854    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139859    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139863    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139867    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139870    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139872    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139875    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139880    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139882    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139885    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139889    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139891    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139894    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139897    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:42.688448 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139900    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139903    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139905    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139908    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139911    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139916    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139918    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139922    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139925    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139928    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139930    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139934    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139938    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139940    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139944    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139947    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139950    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139953    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139957    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139960    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:42.689011 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139963    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139965    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139968    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139970    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139974    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139978    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139980    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139983    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139986    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139990    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139992    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139995    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.139998    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140001    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140004    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140007    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140010    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140014    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140016    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140020    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:42.847100 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140023    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140025    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140028    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140031    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140034    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140037    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140040    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140043    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140046    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140048    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140053    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140056    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140059    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140061    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140066    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140070    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140073    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140076    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140080    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:42.848435 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.140082    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.140798    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.148023    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.148039    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.148057    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.148062    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148125    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148130    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148134    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148136    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148139    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148141    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148144    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148147    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148149    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:42.849124 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148152    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148154    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148157    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148166    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148169    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148172    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148175    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148177    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148180    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148183    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148186    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148190    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148193    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148196    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148198    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148216    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148219    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148222    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148225    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:42.849643 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148227    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148230    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148233    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148237    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148241    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148244    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148247    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148250    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148253    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148255    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148258    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148261    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148263    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148272    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148275    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148278    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148280    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148289    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148292    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148295    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:42.850237 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148298    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148300    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148303    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148305    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148307    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148310    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148313    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148315    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148318    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148320    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148323    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148325    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148328    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148331    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148336    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148339    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148342    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148344    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148347    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:42.850991 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148350    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148352    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148355    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148361    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148364    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148368    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148371    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148374    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148377    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148380    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148383    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148391    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148394    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148397    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148399    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148402    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148404    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148407    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148409    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:42.851553 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148412    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148415    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148417    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148420    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148422    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148425    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148428    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148430    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148435    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148438    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148440    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148443    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148445    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.148451    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148595    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148599    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:42.852060 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148602    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148605    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148607    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148610    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148613    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148615    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148617    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148620    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148623    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148632    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148635    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148637    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148640    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148642    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148645    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148647    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148650    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148652    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148655    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148657    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:42.852501 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148660    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148663    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148667    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148669    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148672    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148676    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148678    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148681    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148683    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148686    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148689    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148691    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148694    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148696    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148699    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148702    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148704    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148706    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148709    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148711    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:42.853039 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148714    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148716    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148724    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148727    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148730    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148732    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148735    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148737    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148739    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148742    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148744    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148747    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148750    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148754    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148758    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148761    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148763    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148768    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148770    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:42.853584 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148773    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148776    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148780    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148786    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148790    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148795    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148799    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148802    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148805    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148807    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148810    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148812    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148815    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148818    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148820    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148822    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148832    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148835    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148837    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:42.854103 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148840    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148844    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148847    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148851    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148855    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148860    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148864    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148868    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148872    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148876    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148878    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148882    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148884    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148887    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148889    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148894    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148897    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148900    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:42.854671 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:42.148903    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.148908    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.149013    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.153446    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.154855    2581 server.go:1019] "Starting client certificate rotation"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.154948    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.154986    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.158288    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.159390    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.171824    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.173832    2581 log.go:25] "Validated CRI v1 image API"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.174655    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 04:58:42.855182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.179415    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 04:58:42.855541 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.180149    2581 fs.go:135] Filesystem UUIDs: map[53273374-f8d8-480b-8444-3f461612b894:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 80b290c7-87d5-43bb-8b8b-a07a54545d15:/dev/nvme0n1p4]
Jul 15 04:58:42.855541 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.180168    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.186239    2581 manager.go:216] Machine: {Timestamp:2026-07-15 04:58:42.184723631 +0000 UTC m=+0.352320294 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3114838 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec294ab4901d8b07cfe23b41104819ae SystemUUID:ec294ab4-901d-8b07-cfe2-3b41104819ae BootID:d6e970db-291f-4136-bf13-d40f91f39f72 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:85:4c:0c:42:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:85:4c:0c:42:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:87:f5:99:07:4d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.186332    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.186403    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.189296    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.189321    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-29.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.190706    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.190716    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.190723    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.191381    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.192693    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198328    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198436    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198461    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198478    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198491    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198503    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198512    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 04:58:42.855606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.198994    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.199022    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.201753    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n585z"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.202470    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.204936    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.204965    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205127    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205140    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205146    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205151    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205160    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205168    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205634    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205641    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205649    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205654    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205663    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.205671    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.206409    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.206420    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.206948    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n585z"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.208949    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-29.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.209031    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-29.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.209123    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.209790    2581 server.go:1268] "Started kubelet"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.210118    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.210549    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.210613    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.211978    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.212241    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.216815    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.217225    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.217241    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.217282    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.217896    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.217896    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.217919    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218042    2581 factory.go:55] Registering systemd factory
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218052    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218066    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218056    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218296    2581 factory.go:153] Registering CRI-O factory
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218310    2581 factory.go:223] Registration of the crio container factory successfully
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218358    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218379    2581 factory.go:103] Registering Raw factory
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218389    2581 manager.go:1196] Started watching for new ooms in manager
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.218920    2581 manager.go:349] Starting recovery of all containers
Jul 15 04:58:42.856117 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.220916    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.228359    2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-29.ec2.internal\" not found" node="ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.233670    2581 manager.go:354] Recovery completed
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.239317    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.241380    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.241410    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.241420    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.241955    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.241967    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.241972    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.241986    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.242716    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.242741    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.242751    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.243721    2581 policy_none.go:49] "None policy: Start"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.243734    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.243743    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.244748    2581 policy_none.go:47] "Start"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.296360    2581 manager.go:337] "Starting Device Plugin manager"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.296472    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.296485    2581 server.go:87] "Starting device plugin registration server"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.296692    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.296711    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.296804    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.296881    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.296887    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.297541    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.297582    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.380075    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.381244    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.381287    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.381326    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.381366    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.383354    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.397374    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.398150    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.398184    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.398209    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.398243    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.407671    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.407689    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-29.ec2.internal\": node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.421716    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.481767    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"]
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.481824    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.482599    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.482630    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.482643    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:42.857388 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484167    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484313    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484861    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484872    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484889    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484896    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484929    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.484900    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.486381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.486402    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.487108    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.487137    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.487154    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.519577    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.519604    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.519625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46ea3c93a970f7700a2a3849415d7a22-config\") pod \"kube-apiserver-proxy-ip-10-0-130-29.ec2.internal\" (UID: \"46ea3c93a970f7700a2a3849415d7a22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.521774    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.523892    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-29.ec2.internal\" not found" node="ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.533017    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-29.ec2.internal\" not found" node="ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.619996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46ea3c93a970f7700a2a3849415d7a22-config\") pod \"kube-apiserver-proxy-ip-10-0-130-29.ec2.internal\" (UID: \"46ea3c93a970f7700a2a3849415d7a22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.620019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.620037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.620082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.858653 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.620102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46ea3c93a970f7700a2a3849415d7a22-config\") pod \"kube-apiserver-proxy-ip-10-0-130-29.ec2.internal\" (UID: \"46ea3c93a970f7700a2a3849415d7a22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.859322 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.620119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.859322 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.622245    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.859322 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.722419    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.859322 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.824384    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:42.859322 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.825609    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.859322 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:42.836223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:42.925432 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:42.925402    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:43.026526 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:43.026475    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:43.126617 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:43.126598    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:43.154267 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.154243    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 04:58:43.154421 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.154398    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 04:58:43.154474 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.154424    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 04:58:43.211092 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.211063    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 04:53:42 +0000 UTC" deadline="2027-12-10 04:02:45.818721709 +0000 UTC"
Jul 15 04:58:43.211092 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.211088    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12311h4m2.607636256s"
Jul 15 04:58:43.227197 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:43.227172    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 04:58:43.307725 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.307673    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:43.315436 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.315418    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:43.317524 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.317507    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:43.325886 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.325863    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 04:58:43.326936 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.326912    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 04:58:43.337547 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.337521    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 04:58:43.424533 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:43.424501    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e866a2a89b76039c681f32045b5640c.slice/crio-42aceded809b1ff72c02f6faf823d435bbce932cd51ae640cabe975c73a7daa7 WatchSource:0}: Error finding container 42aceded809b1ff72c02f6faf823d435bbce932cd51ae640cabe975c73a7daa7: Status 404 returned error can't find the container with id 42aceded809b1ff72c02f6faf823d435bbce932cd51ae640cabe975c73a7daa7
Jul 15 04:58:43.424765 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:43.424745    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ea3c93a970f7700a2a3849415d7a22.slice/crio-16879025cf366f91262d3b9e2f553fb22f9d8d703f0c3a53ec655f8f6d10be45 WatchSource:0}: Error finding container 16879025cf366f91262d3b9e2f553fb22f9d8d703f0c3a53ec655f8f6d10be45: Status 404 returned error can't find the container with id 16879025cf366f91262d3b9e2f553fb22f9d8d703f0c3a53ec655f8f6d10be45
Jul 15 04:58:43.428921 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:43.428907    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:58:44.184708 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.184675    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:44.199934 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.199906    2581 apiserver.go:52] "Watching apiserver"
Jul 15 04:58:44.208837 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.208815    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 04:58:44.210362 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.210327    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal","openshift-cluster-node-tuning-operator/tuned-45gjd","openshift-image-registry/node-ca-n2nlx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal","openshift-multus/multus-k75zq","openshift-network-diagnostics/network-check-target-65s67","kube-system/global-pull-secret-syncer-tq7nn","kube-system/konnectivity-agent-lcwg8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl","openshift-dns/node-resolver-7ndr7","openshift-multus/multus-additional-cni-plugins-2f9m2","openshift-multus/network-metrics-daemon-nx4b2","openshift-network-operator/iptables-alerter-ftzlq","openshift-ovn-kubernetes/ovnkube-node-llwcd"]
Jul 15 04:58:44.212315 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.212288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.214054 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.214031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.215098 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.215077    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 04:58:44.215230 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.215197    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lvkcd\""
Jul 15 04:58:44.215344 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.215327    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 04:58:44.215501 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.215478    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.216527 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.216503    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 04:58:44.216843 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.216736    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 04:58:44.217134 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.217116    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k9fk8\""
Jul 15 04:58:44.217405 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.217386    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 04:58:44.217874 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.217853    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 04:58:44.217971 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.217939    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 04:58:44.218249 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.218228    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 04:58:44.218397 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.218258    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-crgbt\""
Jul 15 04:58:44.219509 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.218497    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 04:58:44.219509 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.218506    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 04:58:44.219509 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.219262    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:44.219509 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.219347    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:44.219509 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.219395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.219509 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.219456    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:44.220340 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.220285    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.222075 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.221779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.222931 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.222675    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 04:58:44.222931 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.222775    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 04:58:44.222931 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.222851    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l6gdd\""
Jul 15 04:58:44.225285 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.224689    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 04:58:44.225285 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.224713    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 04:58:44.225285 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.225046    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 04:58:44.225285 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.225097    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fbn89\""
Jul 15 04:58:44.225544 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.225337    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.226244 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.226205    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.227714 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.227524    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 04:58:44.227714 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.227591    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:44.227714 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.227649    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:44.229331 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.228914    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5tz74\""
Jul 15 04:58:44.229331 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.228951    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 04:58:44.229500 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.229472    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 04:58:44.229598 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.229518    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 04:58:44.229915 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.229895    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b8twq\""
Jul 15 04:58:44.230047 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.230024    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 04:58:44.230588 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.230571    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f13133e-0663-4a5a-b4d6-9a4e14609a76-kubelet-config\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysctl-conf\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-cnibin\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-socket-dir-parent\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-kubelet\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-conf-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gql45\" (UniqueName: \"kubernetes.io/projected/012027f8-f425-4989-ac09-9f8f03a78849-kube-api-access-gql45\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012027f8-f425-4989-ac09-9f8f03a78849-cni-binary-copy\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.233302 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.232994    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 04:58:44.233862 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.233705    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wdjp5\""
Jul 15 04:58:44.233862 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.233714    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 04:58:44.234069 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.233866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.234069 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysctl-d\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.234164 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-systemd\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.234832 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234695    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 04:58:44.234832 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234696    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-tuned\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.234832 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ac8bdf1-c573-43fa-addc-540578fcb845-tmp\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.235044 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-os-release\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235044 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-hostroot\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235044 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.234971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-modprobe-d\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.235044 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdknp\" (UniqueName: \"kubernetes.io/projected/9ac8bdf1-c573-43fa-addc-540578fcb845-kube-api-access-cdknp\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.235245 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfmck\" (UniqueName: \"kubernetes.io/projected/321e957c-8792-4fa3-974d-e9352212e364-kube-api-access-dfmck\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.235245 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-k8s-cni-cncf-io\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235245 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-multus-certs\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235435 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:44.235435 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-host\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.235435 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235358    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/321e957c-8792-4fa3-974d-e9352212e364-serviceca\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.235435 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-system-cni-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235646 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-cni-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235646 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-run\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.235646 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-var-lib-kubelet\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.235646 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/012027f8-f425-4989-ac09-9f8f03a78849-multus-daemon-config\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235646 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f13133e-0663-4a5a-b4d6-9a4e14609a76-dbus\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.235908 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-netns\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235908 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-cni-bin\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235908 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-cni-multus\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235908 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-etc-kubernetes\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.235908 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-kubernetes\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.235908 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-sys\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.236264 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysconfig\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.236264 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-lib-modules\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.236264 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.235991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/321e957c-8792-4fa3-974d-e9352212e364-host\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.236859 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.236839    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 04:58:44.237085 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.237063    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 04:58:44.237178 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.236778    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 04:58:44.237511 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.237493    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4szt8\""
Jul 15 04:58:44.237604 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.237513    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 04:58:44.237665 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.237647    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 04:58:44.237788 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.237736    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 04:58:44.253541 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.253522    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vgdvq"
Jul 15 04:58:44.261528 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.261501    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vgdvq"
Jul 15 04:58:44.264795 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.264761    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:44.319631 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.319607    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 04:58:44.336929 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.336899    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.337054 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.336949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f13133e-0663-4a5a-b4d6-9a4e14609a76-dbus\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.337054 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.336980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-netns\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337054 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-cni-bin\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337054 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-cni-multus\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-netns\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-cni-multus\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-cni-bin\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-etc-kubernetes\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-sys-fs\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337237    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-etc-kubernetes\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1105801c-45da-4220-a30a-84dcc66eca73-iptables-alerter-script\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-kubernetes\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.337289 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337251    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f13133e-0663-4a5a-b4d6-9a4e14609a76-dbus\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-tmp-dir\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-var-lib-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-kubernetes\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-ovnkube-config\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/216bc077-7039-40ec-b23c-b3737183c581-ovn-node-metrics-cert\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysconfig\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-device-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysconfig\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-hosts-file\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-kubelet\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-systemd-units\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.337694 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-run-ovn-kubernetes\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-kubelet\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-ovn\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvbwj\" (UniqueName: \"kubernetes.io/projected/216bc077-7039-40ec-b23c-b3737183c581-kube-api-access-tvbwj\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-var-lib-kubelet\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-cni-binary-copy\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvqgn\" (UniqueName: \"kubernetes.io/projected/1105801c-45da-4220-a30a-84dcc66eca73-kube-api-access-bvqgn\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012027f8-f425-4989-ac09-9f8f03a78849-cni-binary-copy\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.337949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-tuned\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.338129    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-os-release\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-hostroot\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.338384 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338352    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-os-release\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.338499    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret podName:6f13133e-0663-4a5a-b4d6-9a4e14609a76 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:44.838457647 +0000 UTC m=+3.006054302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret") pod "global-pull-secret-syncer-tq7nn" (UID: "6f13133e-0663-4a5a-b4d6-9a4e14609a76") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-hostroot\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-systemd\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-etc-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdknp\" (UniqueName: \"kubernetes.io/projected/9ac8bdf1-c573-43fa-addc-540578fcb845-kube-api-access-cdknp\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338625    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-k8s-cni-cncf-io\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-cni-bin\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-cni-netd\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/012027f8-f425-4989-ac09-9f8f03a78849-cni-binary-copy\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-host\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/321e957c-8792-4fa3-974d-e9352212e364-serviceca\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-k8s-cni-cncf-io\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338856    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-system-cni-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-system-cni-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-host\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.339014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-cni-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-registration-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-system-cni-dir\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.338989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-cni-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-cnibin\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-sys\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-node-log\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-sys\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339120    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhplf\" (UniqueName: \"kubernetes.io/projected/0d613161-cebc-4757-b35a-ddfb39c8501b-kube-api-access-rhplf\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpr29\" (UniqueName: \"kubernetes.io/projected/971ca253-982f-4e83-86ad-cba13a9970e0-kube-api-access-fpr29\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-lib-modules\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/321e957c-8792-4fa3-974d-e9352212e364-host\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-socket-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgg5t\" (UniqueName: \"kubernetes.io/projected/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-kube-api-access-mgg5t\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/321e957c-8792-4fa3-974d-e9352212e364-serviceca\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-slash\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.339794 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-lib-modules\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-log-socket\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/321e957c-8792-4fa3-974d-e9352212e364-host\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-ovnkube-script-lib\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f13133e-0663-4a5a-b4d6-9a4e14609a76-kubelet-config\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysctl-conf\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-cnibin\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-socket-dir-parent\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-conf-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339647    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f13133e-0663-4a5a-b4d6-9a4e14609a76-kubelet-config\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gql45\" (UniqueName: \"kubernetes.io/projected/012027f8-f425-4989-ac09-9f8f03a78849-kube-api-access-gql45\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-cnibin\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-env-overrides\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-multus-certs\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1105801c-45da-4220-a30a-84dcc66eca73-host-slash\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.340560 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysctl-d\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysctl-conf\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-systemd\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-socket-dir-parent\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339921    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-systemd\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ac8bdf1-c573-43fa-addc-540578fcb845-tmp\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-run-netns\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.339992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-host-run-multus-certs\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/012027f8-f425-4989-ac09-9f8f03a78849-multus-conf-dir\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-sysctl-d\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-os-release\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-modprobe-d\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfmck\" (UniqueName: \"kubernetes.io/projected/321e957c-8792-4fa3-974d-e9352212e364-kube-api-access-dfmck\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2941fa73-a703-4c81-b68b-a1c89a7cd65d-konnectivity-ca\") pod \"konnectivity-agent-lcwg8\" (UID: \"2941fa73-a703-4c81-b68b-a1c89a7cd65d\") " pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7xsv\" (UniqueName: \"kubernetes.io/projected/0f2506dd-dda6-410d-8881-397763b92799-kube-api-access-n7xsv\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-run\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.341430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-var-lib-kubelet\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.342161 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340382    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-modprobe-d\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.342161 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/012027f8-f425-4989-ac09-9f8f03a78849-multus-daemon-config\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.342161 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2941fa73-a703-4c81-b68b-a1c89a7cd65d-agent-certs\") pod \"konnectivity-agent-lcwg8\" (UID: \"2941fa73-a703-4c81-b68b-a1c89a7cd65d\") " pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.342161 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-run\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.342161 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.340576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ac8bdf1-c573-43fa-addc-540578fcb845-var-lib-kubelet\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.342161 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.341026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/012027f8-f425-4989-ac09-9f8f03a78849-multus-daemon-config\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.342576 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.342557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9ac8bdf1-c573-43fa-addc-540578fcb845-etc-tuned\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.342727 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.342705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ac8bdf1-c573-43fa-addc-540578fcb845-tmp\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.348334 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.348316    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:44.348334 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.348337    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:44.348555 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.348351    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hf76r for pod openshift-network-diagnostics/network-check-target-65s67: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:44.348555 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.348411    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r podName:8d1616bb-ada0-4fdb-8b93-1ea20433aa69 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:44.848393696 +0000 UTC m=+3.015990366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hf76r" (UniqueName: "kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r") pod "network-check-target-65s67" (UID: "8d1616bb-ada0-4fdb-8b93-1ea20433aa69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:44.351687 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.351369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gql45\" (UniqueName: \"kubernetes.io/projected/012027f8-f425-4989-ac09-9f8f03a78849-kube-api-access-gql45\") pod \"multus-k75zq\" (UID: \"012027f8-f425-4989-ac09-9f8f03a78849\") " pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.352729 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.352686    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfmck\" (UniqueName: \"kubernetes.io/projected/321e957c-8792-4fa3-974d-e9352212e364-kube-api-access-dfmck\") pod \"node-ca-n2nlx\" (UID: \"321e957c-8792-4fa3-974d-e9352212e364\") " pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.354437 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.354392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdknp\" (UniqueName: \"kubernetes.io/projected/9ac8bdf1-c573-43fa-addc-540578fcb845-kube-api-access-cdknp\") pod \"tuned-45gjd\" (UID: \"9ac8bdf1-c573-43fa-addc-540578fcb845\") " pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.388807 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.388764    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" event={"ID":"4e866a2a89b76039c681f32045b5640c","Type":"ContainerStarted","Data":"42aceded809b1ff72c02f6faf823d435bbce932cd51ae640cabe975c73a7daa7"}
Jul 15 04:58:44.391630 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.391607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal" event={"ID":"46ea3c93a970f7700a2a3849415d7a22","Type":"ContainerStarted","Data":"16879025cf366f91262d3b9e2f553fb22f9d8d703f0c3a53ec655f8f6d10be45"}
Jul 15 04:58:44.441672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-node-log\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.441672 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhplf\" (UniqueName: \"kubernetes.io/projected/0d613161-cebc-4757-b35a-ddfb39c8501b-kube-api-access-rhplf\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpr29\" (UniqueName: \"kubernetes.io/projected/971ca253-982f-4e83-86ad-cba13a9970e0-kube-api-access-fpr29\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-node-log\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-socket-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgg5t\" (UniqueName: \"kubernetes.io/projected/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-kube-api-access-mgg5t\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-slash\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-log-socket\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441830    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-ovnkube-script-lib\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.441860 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-log-socket\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-env-overrides\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-slash\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1105801c-45da-4220-a30a-84dcc66eca73-host-slash\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.441986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-run-netns\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-os-release\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2941fa73-a703-4c81-b68b-a1c89a7cd65d-konnectivity-ca\") pod \"konnectivity-agent-lcwg8\" (UID: \"2941fa73-a703-4c81-b68b-a1c89a7cd65d\") " pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-socket-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7xsv\" (UniqueName: \"kubernetes.io/projected/0f2506dd-dda6-410d-8881-397763b92799-kube-api-access-n7xsv\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2941fa73-a703-4c81-b68b-a1c89a7cd65d-agent-certs\") pod \"konnectivity-agent-lcwg8\" (UID: \"2941fa73-a703-4c81-b68b-a1c89a7cd65d\") " pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-os-release\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-sys-fs\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.442336 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1105801c-45da-4220-a30a-84dcc66eca73-iptables-alerter-script\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-tmp-dir\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-var-lib-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-ovnkube-config\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/216bc077-7039-40ec-b23c-b3737183c581-ovn-node-metrics-cert\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-device-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442590    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-hosts-file\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-var-lib-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-ovnkube-script-lib\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442604    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-env-overrides\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-kubelet\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-sys-fs\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-run-netns\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.443223 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2941fa73-a703-4c81-b68b-a1c89a7cd65d-konnectivity-ca\") pod \"konnectivity-agent-lcwg8\" (UID: \"2941fa73-a703-4c81-b68b-a1c89a7cd65d\") " pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1105801c-45da-4220-a30a-84dcc66eca73-host-slash\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-kubelet\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-device-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.442821    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-hosts-file\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-systemd-units\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-systemd-units\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-run-ovn-kubernetes\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-run-ovn-kubernetes\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1105801c-45da-4220-a30a-84dcc66eca73-iptables-alerter-script\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-ovn\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvbwj\" (UniqueName: \"kubernetes.io/projected/216bc077-7039-40ec-b23c-b3737183c581-kube-api-access-tvbwj\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-ovn\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/216bc077-7039-40ec-b23c-b3737183c581-ovnkube-config\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444129 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-cni-binary-copy\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvqgn\" (UniqueName: \"kubernetes.io/projected/1105801c-45da-4220-a30a-84dcc66eca73-kube-api-access-bvqgn\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-tmp-dir\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-systemd\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-etc-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-cni-bin\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-run-systemd\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-cni-netd\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-registration-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-system-cni-dir\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-cnibin\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f2506dd-dda6-410d-8881-397763b92799-registration-dir\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.443852    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-cni-bin\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-system-cni-dir\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.444768 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-etc-openvswitch\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.445601 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.443927    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:44.943908269 +0000 UTC m=+3.111504934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:44.445601 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/216bc077-7039-40ec-b23c-b3737183c581-host-cni-netd\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.445601 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.445601 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443957    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d613161-cebc-4757-b35a-ddfb39c8501b-cnibin\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.445601 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.443983    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d613161-cebc-4757-b35a-ddfb39c8501b-cni-binary-copy\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.445970 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.445940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2941fa73-a703-4c81-b68b-a1c89a7cd65d-agent-certs\") pod \"konnectivity-agent-lcwg8\" (UID: \"2941fa73-a703-4c81-b68b-a1c89a7cd65d\") " pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.446197 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.446174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/216bc077-7039-40ec-b23c-b3737183c581-ovn-node-metrics-cert\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.453981 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.453929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhplf\" (UniqueName: \"kubernetes.io/projected/0d613161-cebc-4757-b35a-ddfb39c8501b-kube-api-access-rhplf\") pod \"multus-additional-cni-plugins-2f9m2\" (UID: \"0d613161-cebc-4757-b35a-ddfb39c8501b\") " pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.454456 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.454417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7xsv\" (UniqueName: \"kubernetes.io/projected/0f2506dd-dda6-410d-8881-397763b92799-kube-api-access-n7xsv\") pod \"aws-ebs-csi-driver-node-ghjbl\" (UID: \"0f2506dd-dda6-410d-8881-397763b92799\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.454565 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.454548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgg5t\" (UniqueName: \"kubernetes.io/projected/f9d9807f-4795-4f9c-8ceb-e41e98fdfed0-kube-api-access-mgg5t\") pod \"node-resolver-7ndr7\" (UID: \"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0\") " pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.455099 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.455078    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpr29\" (UniqueName: \"kubernetes.io/projected/971ca253-982f-4e83-86ad-cba13a9970e0-kube-api-access-fpr29\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:44.455783 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.455759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvqgn\" (UniqueName: \"kubernetes.io/projected/1105801c-45da-4220-a30a-84dcc66eca73-kube-api-access-bvqgn\") pod \"iptables-alerter-ftzlq\" (UID: \"1105801c-45da-4220-a30a-84dcc66eca73\") " pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.455875 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.455835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvbwj\" (UniqueName: \"kubernetes.io/projected/216bc077-7039-40ec-b23c-b3737183c581-kube-api-access-tvbwj\") pod \"ovnkube-node-llwcd\" (UID: \"216bc077-7039-40ec-b23c-b3737183c581\") " pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.527929 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.527904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-45gjd"
Jul 15 04:58:44.538349 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.538328    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n2nlx"
Jul 15 04:58:44.548440 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.548421    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k75zq"
Jul 15 04:58:44.554491 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.554472    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:58:44.563262 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.563245    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl"
Jul 15 04:58:44.578242 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.578194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7ndr7"
Jul 15 04:58:44.586090 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.586070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2f9m2"
Jul 15 04:58:44.595065 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.595047    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ftzlq"
Jul 15 04:58:44.601913 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.601896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:58:44.846542 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.846501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:44.846686 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.846628    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:44.846741 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.846688    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret podName:6f13133e-0663-4a5a-b4d6-9a4e14609a76 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:45.846673756 +0000 UTC m=+4.014270408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret") pod "global-pull-secret-syncer-tq7nn" (UID: "6f13133e-0663-4a5a-b4d6-9a4e14609a76") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:44.947067 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.947034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:44.947288 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:44.947129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:44.947288 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.947239    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:44.947288 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.947271    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:44.947475 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.947309    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hf76r for pod openshift-network-diagnostics/network-check-target-65s67: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:44.947475 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.947371    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r podName:8d1616bb-ada0-4fdb-8b93-1ea20433aa69 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:45.947350441 +0000 UTC m=+4.114947113 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf76r" (UniqueName: "kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r") pod "network-check-target-65s67" (UID: "8d1616bb-ada0-4fdb-8b93-1ea20433aa69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:44.947475 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.947271    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:44.947475 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:44.947454    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:45.947432382 +0000 UTC m=+4.115029053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:45.264545 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.264451    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 04:53:44 +0000 UTC" deadline="2028-01-29 05:30:06.695476506 +0000 UTC"
Jul 15 04:58:45.264545 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.264484    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13512h31m21.430996638s"
Jul 15 04:58:45.381595 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.381569    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:45.382035 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.381632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:45.382035 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.381720    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:45.382035 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.381740    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:45.382035 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.381801    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:45.382035 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.381934    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:45.393283 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.393204    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d9807f_4795_4f9c_8ceb_e41e98fdfed0.slice/crio-f5e3559d5d4c3f15fe5ec5bd03ee7b0e0cc52e1031ce36903e44f8e276778325 WatchSource:0}: Error finding container f5e3559d5d4c3f15fe5ec5bd03ee7b0e0cc52e1031ce36903e44f8e276778325: Status 404 returned error can't find the container with id f5e3559d5d4c3f15fe5ec5bd03ee7b0e0cc52e1031ce36903e44f8e276778325
Jul 15 04:58:45.394862 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.394780    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2941fa73_a703_4c81_b68b_a1c89a7cd65d.slice/crio-3ea57881569cdbddd6226753017fc9b2184a081c07f8afd3c31821c6a353b9e9 WatchSource:0}: Error finding container 3ea57881569cdbddd6226753017fc9b2184a081c07f8afd3c31821c6a353b9e9: Status 404 returned error can't find the container with id 3ea57881569cdbddd6226753017fc9b2184a081c07f8afd3c31821c6a353b9e9
Jul 15 04:58:45.399385 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.399355    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac8bdf1_c573_43fa_addc_540578fcb845.slice/crio-aa199531174bb2952082e7282052bd1e33255eba1da674f481d4a6b702573a83 WatchSource:0}: Error finding container aa199531174bb2952082e7282052bd1e33255eba1da674f481d4a6b702573a83: Status 404 returned error can't find the container with id aa199531174bb2952082e7282052bd1e33255eba1da674f481d4a6b702573a83
Jul 15 04:58:45.400151 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.400128    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321e957c_8792_4fa3_974d_e9352212e364.slice/crio-a410b6e1035619dd7866ee9cbcf75a60f78b0edc1a993da32b2d5878bf0fe5d9 WatchSource:0}: Error finding container a410b6e1035619dd7866ee9cbcf75a60f78b0edc1a993da32b2d5878bf0fe5d9: Status 404 returned error can't find the container with id a410b6e1035619dd7866ee9cbcf75a60f78b0edc1a993da32b2d5878bf0fe5d9
Jul 15 04:58:45.401203 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.401091    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2506dd_dda6_410d_8881_397763b92799.slice/crio-f74c1da7d89484b8e24907996dccfec2cc24af548671ff7beabc0eb372978d45 WatchSource:0}: Error finding container f74c1da7d89484b8e24907996dccfec2cc24af548671ff7beabc0eb372978d45: Status 404 returned error can't find the container with id f74c1da7d89484b8e24907996dccfec2cc24af548671ff7beabc0eb372978d45
Jul 15 04:58:45.403004 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.402895    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216bc077_7039_40ec_b23c_b3737183c581.slice/crio-fa18f5de5efc4e3e4aa70ecec0f64e19856391a616fe5d0d1f4eb804399d6cc6 WatchSource:0}: Error finding container fa18f5de5efc4e3e4aa70ecec0f64e19856391a616fe5d0d1f4eb804399d6cc6: Status 404 returned error can't find the container with id fa18f5de5efc4e3e4aa70ecec0f64e19856391a616fe5d0d1f4eb804399d6cc6
Jul 15 04:58:45.404587 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.404487    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1105801c_45da_4220_a30a_84dcc66eca73.slice/crio-80b45470fcb22f5f0cce777be0f6d794a3e8bb5878e2aca467a9b8422b20e422 WatchSource:0}: Error finding container 80b45470fcb22f5f0cce777be0f6d794a3e8bb5878e2aca467a9b8422b20e422: Status 404 returned error can't find the container with id 80b45470fcb22f5f0cce777be0f6d794a3e8bb5878e2aca467a9b8422b20e422
Jul 15 04:58:45.406430 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:58:45.406407    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012027f8_f425_4989_ac09_9f8f03a78849.slice/crio-adee15383a1fbe551542a832b653854d92fb7abfac8d7b8747f28fd79bb5ba85 WatchSource:0}: Error finding container adee15383a1fbe551542a832b653854d92fb7abfac8d7b8747f28fd79bb5ba85: Status 404 returned error can't find the container with id adee15383a1fbe551542a832b653854d92fb7abfac8d7b8747f28fd79bb5ba85
Jul 15 04:58:45.854970 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.854623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:45.855130 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.855084    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:45.855202 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.855151    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret podName:6f13133e-0663-4a5a-b4d6-9a4e14609a76 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:47.855131013 +0000 UTC m=+6.022727672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret") pod "global-pull-secret-syncer-tq7nn" (UID: "6f13133e-0663-4a5a-b4d6-9a4e14609a76") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:45.955699 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.955657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:45.955887 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:45.955731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:45.955969 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.955925    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:45.955969 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.955948    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:45.955969 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.955962    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hf76r for pod openshift-network-diagnostics/network-check-target-65s67: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:45.956127 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.956025    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r podName:8d1616bb-ada0-4fdb-8b93-1ea20433aa69 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:47.956006402 +0000 UTC m=+6.123603057 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf76r" (UniqueName: "kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r") pod "network-check-target-65s67" (UID: "8d1616bb-ada0-4fdb-8b93-1ea20433aa69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:45.956515 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.956494    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:45.956599 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:45.956555    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:47.956538815 +0000 UTC m=+6.124135482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:46.265423 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.265288    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 04:53:44 +0000 UTC" deadline="2028-03-07 05:25:59.475452928 +0000 UTC"
Jul 15 04:58:46.265423 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.265346    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14424h27m13.210111102s"
Jul 15 04:58:46.402620 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.402557    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerStarted","Data":"efc6a82ed464559d2793d2337ed96d399718db22d728bc9208972a5ecc81f5ff"}
Jul 15 04:58:46.405348 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.405320    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n2nlx" event={"ID":"321e957c-8792-4fa3-974d-e9352212e364","Type":"ContainerStarted","Data":"a410b6e1035619dd7866ee9cbcf75a60f78b0edc1a993da32b2d5878bf0fe5d9"}
Jul 15 04:58:46.419135 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.419105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal" event={"ID":"46ea3c93a970f7700a2a3849415d7a22","Type":"ContainerStarted","Data":"ee09db994f0d7a34625c1bb8367408c7638b89cbe9c6ea02f15a1918c28646ab"}
Jul 15 04:58:46.425820 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.425754    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k75zq" event={"ID":"012027f8-f425-4989-ac09-9f8f03a78849","Type":"ContainerStarted","Data":"adee15383a1fbe551542a832b653854d92fb7abfac8d7b8747f28fd79bb5ba85"}
Jul 15 04:58:46.432841 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.432759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ftzlq" event={"ID":"1105801c-45da-4220-a30a-84dcc66eca73","Type":"ContainerStarted","Data":"80b45470fcb22f5f0cce777be0f6d794a3e8bb5878e2aca467a9b8422b20e422"}
Jul 15 04:58:46.439853 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.439231    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal" podStartSLOduration=3.439212761 podStartE2EDuration="3.439212761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:58:46.438543059 +0000 UTC m=+4.606139741" watchObservedRunningTime="2026-07-15 04:58:46.439212761 +0000 UTC m=+4.606809437"
Jul 15 04:58:46.440325 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.440201    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"fa18f5de5efc4e3e4aa70ecec0f64e19856391a616fe5d0d1f4eb804399d6cc6"}
Jul 15 04:58:46.447642 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.447559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl" event={"ID":"0f2506dd-dda6-410d-8881-397763b92799","Type":"ContainerStarted","Data":"f74c1da7d89484b8e24907996dccfec2cc24af548671ff7beabc0eb372978d45"}
Jul 15 04:58:46.455202 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.455128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-45gjd" event={"ID":"9ac8bdf1-c573-43fa-addc-540578fcb845","Type":"ContainerStarted","Data":"aa199531174bb2952082e7282052bd1e33255eba1da674f481d4a6b702573a83"}
Jul 15 04:58:46.464765 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.464737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lcwg8" event={"ID":"2941fa73-a703-4c81-b68b-a1c89a7cd65d","Type":"ContainerStarted","Data":"3ea57881569cdbddd6226753017fc9b2184a081c07f8afd3c31821c6a353b9e9"}
Jul 15 04:58:46.472073 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.472000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7ndr7" event={"ID":"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0","Type":"ContainerStarted","Data":"f5e3559d5d4c3f15fe5ec5bd03ee7b0e0cc52e1031ce36903e44f8e276778325"}
Jul 15 04:58:46.476353 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.475714    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e866a2a89b76039c681f32045b5640c" containerID="ce0936f276c3283b423308e4b2a44e3a9fe70cc6204b2c6b5ef6472ddf6583ea" exitCode=0
Jul 15 04:58:46.476353 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:46.475748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" event={"ID":"4e866a2a89b76039c681f32045b5640c","Type":"ContainerDied","Data":"ce0936f276c3283b423308e4b2a44e3a9fe70cc6204b2c6b5ef6472ddf6583ea"}
Jul 15 04:58:47.383326 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.382520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:47.383326 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.382643    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:47.383326 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.382727    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:47.383326 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.382814    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:47.383326 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.382869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:47.383326 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.382940    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:47.490392 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.490353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" event={"ID":"4e866a2a89b76039c681f32045b5640c","Type":"ContainerStarted","Data":"21a00f81e97384bae1d5cb3933a6bb6796701c2bc5e6b246187f074717fdb406"}
Jul 15 04:58:47.509368 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.509172    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" podStartSLOduration=4.509149971 podStartE2EDuration="4.509149971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:58:47.508272901 +0000 UTC m=+5.675869575" watchObservedRunningTime="2026-07-15 04:58:47.509149971 +0000 UTC m=+5.676746645"
Jul 15 04:58:47.871519 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.870849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:47.871519 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.871010    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:47.871519 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.871076    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret podName:6f13133e-0663-4a5a-b4d6-9a4e14609a76 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:51.871055621 +0000 UTC m=+10.038652278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret") pod "global-pull-secret-syncer-tq7nn" (UID: "6f13133e-0663-4a5a-b4d6-9a4e14609a76") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:47.971836 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.971800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:47.972054 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:47.971910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:47.972054 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.971958    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:47.972054 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.971984    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:47.972054 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.972000    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hf76r for pod openshift-network-diagnostics/network-check-target-65s67: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:47.972054 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.972040    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:47.972367 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.972069    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r podName:8d1616bb-ada0-4fdb-8b93-1ea20433aa69 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:51.972048242 +0000 UTC m=+10.139644896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf76r" (UniqueName: "kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r") pod "network-check-target-65s67" (UID: "8d1616bb-ada0-4fdb-8b93-1ea20433aa69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:47.972367 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:47.972094    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:51.972077578 +0000 UTC m=+10.139674237 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:49.383201 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:49.382380    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:49.383201 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:49.382516    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:49.383201 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:49.382920    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:49.383201 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:49.383010    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:49.383201 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:49.383070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:49.383201 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:49.383155    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:51.382159 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:51.382128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:51.382159 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:51.382149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:51.382678 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:51.382128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:51.382678 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:51.382272    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:51.382678 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:51.382339    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:51.382678 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:51.382418    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:51.907486 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:51.907443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:51.907662 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:51.907622    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:51.907730 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:51.907697    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret podName:6f13133e-0663-4a5a-b4d6-9a4e14609a76 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:59.907676915 +0000 UTC m=+18.075273572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret") pod "global-pull-secret-syncer-tq7nn" (UID: "6f13133e-0663-4a5a-b4d6-9a4e14609a76") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:52.008600 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:52.008552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:52.008744 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:52.008641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:52.008788 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:52.008769    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:52.008855 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:52.008830    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:00.008811436 +0000 UTC m=+18.176408109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:52.009239 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:52.008922    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:52.009239 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:52.008944    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:52.009239 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:52.008957    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hf76r for pod openshift-network-diagnostics/network-check-target-65s67: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:52.009239 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:52.008992    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r podName:8d1616bb-ada0-4fdb-8b93-1ea20433aa69 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:00.00898097 +0000 UTC m=+18.176577624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf76r" (UniqueName: "kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r") pod "network-check-target-65s67" (UID: "8d1616bb-ada0-4fdb-8b93-1ea20433aa69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:53.382534 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:53.382030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:53.382534 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:53.382176    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:53.382534 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:53.382217    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:53.382534 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:53.382344    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:53.382534 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:53.382407    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:53.382534 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:53.382496    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:55.382177 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:55.382135    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:55.382660 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:55.382216    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:55.382660 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:55.382154    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:55.382660 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:55.382293    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:55.382660 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:55.382393    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:55.382660 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:55.382476    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:57.382518 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:57.382485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:57.382930 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:57.382486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:57.382930 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:57.382603    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:57.382930 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:57.382688    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:57.382930 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:57.382497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:57.382930 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:57.382805    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:59.382348 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:59.382311    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:58:59.382814 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:59.382367    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:58:59.382814 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:59.382313    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:59.382814 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:59.382456    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:58:59.382814 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:59.382536    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:58:59.382814 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:59.382610    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:58:59.973494 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:58:59.973453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:58:59.973671 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:59.973601    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:59.973719 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:58:59.973677    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret podName:6f13133e-0663-4a5a-b4d6-9a4e14609a76 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:15.973654612 +0000 UTC m=+34.141251264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret") pod "global-pull-secret-syncer-tq7nn" (UID: "6f13133e-0663-4a5a-b4d6-9a4e14609a76") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:59:00.074522 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:00.074494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:00.074698 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:00.074537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:00.074698 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:00.074656    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:59:00.074813 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:00.074728    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:16.074707615 +0000 UTC m=+34.242304281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:59:00.074813 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:00.074766    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:59:00.074813 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:00.074791    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:59:00.074813 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:00.074803    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hf76r for pod openshift-network-diagnostics/network-check-target-65s67: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:59:00.074991 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:00.074856    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r podName:8d1616bb-ada0-4fdb-8b93-1ea20433aa69 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:16.074843169 +0000 UTC m=+34.242439835 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf76r" (UniqueName: "kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r") pod "network-check-target-65s67" (UID: "8d1616bb-ada0-4fdb-8b93-1ea20433aa69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:59:01.381845 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:01.381816    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:01.382322 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:01.381913    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:01.382322 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:01.381920    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:01.382322 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:01.381924    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:01.382322 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:01.382008    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:01.382322 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:01.382088    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:03.382182 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:03.382144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:03.382638 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:03.382144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:03.382638 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:03.382268    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:03.382638 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:03.382388    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:03.382638 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:03.382158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:03.382638 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:03.382505    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:05.382478 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.382303    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:05.383257 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.382314    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:05.383257 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:05.382554    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:05.383257 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:05.382665    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:05.383257 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.382366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:05.383257 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:05.382762    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:05.529798 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.529773    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl" event={"ID":"0f2506dd-dda6-410d-8881-397763b92799","Type":"ContainerStarted","Data":"d729a11beb1a6ee77a463debc0baf37d307ecc10ffb5943ec6b8163259fe51a7"}
Jul 15 04:59:05.531007 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.530989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-45gjd" event={"ID":"9ac8bdf1-c573-43fa-addc-540578fcb845","Type":"ContainerStarted","Data":"bb6c440c4a0460bf96b3d5e365935589d88b7651a3c3203db7934bbef46c53ca"}
Jul 15 04:59:05.532375 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.532352    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lcwg8" event={"ID":"2941fa73-a703-4c81-b68b-a1c89a7cd65d","Type":"ContainerStarted","Data":"f60f0e95b91dcdb49bd2db70cc300282bf012aac2f82df35802a9822d4fbc881"}
Jul 15 04:59:05.533754 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.533732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7ndr7" event={"ID":"f9d9807f-4795-4f9c-8ceb-e41e98fdfed0","Type":"ContainerStarted","Data":"793e0d1fb48ef908a75a4df5c8a7b9fb1c6cbcc30a4a8bfdf35131bd55145472"}
Jul 15 04:59:05.535246 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.535226    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d613161-cebc-4757-b35a-ddfb39c8501b" containerID="62ad71a74c520d0bdc5188cf1703c0cce42bfc206fe1c9684211d2c23e6c9b8e" exitCode=0
Jul 15 04:59:05.535326 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.535304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerDied","Data":"62ad71a74c520d0bdc5188cf1703c0cce42bfc206fe1c9684211d2c23e6c9b8e"}
Jul 15 04:59:05.536854 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.536834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n2nlx" event={"ID":"321e957c-8792-4fa3-974d-e9352212e364","Type":"ContainerStarted","Data":"cf34332d7519c782f553afbfd403edafc9b79d26a94bf624ac5d111191da704c"}
Jul 15 04:59:05.538228 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.538190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k75zq" event={"ID":"012027f8-f425-4989-ac09-9f8f03a78849","Type":"ContainerStarted","Data":"1fd856e774912a847d4cc3d2fd97e5f386ea19d4b1747451f48a5f068c5cc1b2"}
Jul 15 04:59:05.541152 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.541121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"d06484c9d144d5343946f707198e73238b0c7beefa82741bb8a48524eca779ef"}
Jul 15 04:59:05.541252 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.541186    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"f70ce89af7efe17e29d9f7d8768a7d6e2becc60ba14e20e2e6bea8fadc5f1b19"}
Jul 15 04:59:05.541252 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.541244    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"4df29abe7e946112a399e56f0a569b289ad2f8aa54115e3994f2143e94181fed"}
Jul 15 04:59:05.541357 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.541281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"4bb9d574a7a2bb1ff559ee5031385df27f00b73372b79e9c258fca8fcd996a21"}
Jul 15 04:59:05.541357 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.541313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"f3dc2a39860c7ec8e4991579c95208f77da0e99d89e33b4dfca737a538ceb32b"}
Jul 15 04:59:05.541357 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.541341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"2b42d218d2d9419747eabc7fb75984469fbce54314d61d15f060e974ebb9b0af"}
Jul 15 04:59:05.548157 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.548124    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-45gjd" podStartSLOduration=3.531123161 podStartE2EDuration="22.548112024s" totalImagesPullingTime="19.016988863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:05.547653952 +0000 UTC m=+23.715250625" watchObservedRunningTime="2026-07-15 04:59:05.548112024 +0000 UTC m=+23.715708748"
Jul 15 04:59:05.582740 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.582705    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lcwg8" podStartSLOduration=3.62565208 podStartE2EDuration="22.582684495s" totalImagesPullingTime="18.957032415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:05.58209906 +0000 UTC m=+23.749695732" watchObservedRunningTime="2026-07-15 04:59:05.582684495 +0000 UTC m=+23.750281166"
Jul 15 04:59:05.597087 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.597051    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7ndr7" podStartSLOduration=3.608180365 podStartE2EDuration="22.597040051s" totalImagesPullingTime="18.988859686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:05.596785364 +0000 UTC m=+23.764382036" watchObservedRunningTime="2026-07-15 04:59:05.597040051 +0000 UTC m=+23.764636723"
Jul 15 04:59:05.610235 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.610179    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n2nlx" podStartSLOduration=3.628198372 podStartE2EDuration="22.610163289s" totalImagesPullingTime="18.981964917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:05.609758652 +0000 UTC m=+23.777355324" watchObservedRunningTime="2026-07-15 04:59:05.610163289 +0000 UTC m=+23.777759962"
Jul 15 04:59:05.627876 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.627838    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k75zq" podStartSLOduration=3.592848097 podStartE2EDuration="22.62782756s" totalImagesPullingTime="19.034979463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:05.627505001 +0000 UTC m=+23.795101673" watchObservedRunningTime="2026-07-15 04:59:05.62782756 +0000 UTC m=+23.795424228"
Jul 15 04:59:05.703984 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:05.703958    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 04:59:06.311333 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.311186    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T04:59:05.703980182Z","UUID":"82596bf6-b231-46e6-8661-4d59f66cb6e5","Handler":null,"Name":"","Endpoint":""}
Jul 15 04:59:06.313313 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.313286    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 04:59:06.313444 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.313332    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 04:59:06.418565 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.418539    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:59:06.419340 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.419318    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:59:06.545881 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.545847    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ftzlq" event={"ID":"1105801c-45da-4220-a30a-84dcc66eca73","Type":"ContainerStarted","Data":"19735bbcbdf8910d84410d301eb1ee0994d3b8be20d1a14e1cf368d8de1a1fd5"}
Jul 15 04:59:06.548428 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.548390    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl" event={"ID":"0f2506dd-dda6-410d-8881-397763b92799","Type":"ContainerStarted","Data":"5c78f901f538330dfd7188e67e3f8cb494cfaffa7d10dcf8239eb03d1d159f7c"}
Jul 15 04:59:06.549373 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.549229    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:59:06.549373 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.549293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lcwg8"
Jul 15 04:59:06.561219 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:06.561158    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ftzlq" podStartSLOduration=4.578865713 podStartE2EDuration="23.56114131s" totalImagesPullingTime="18.982275597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:06.56054569 +0000 UTC m=+24.728142363" watchObservedRunningTime="2026-07-15 04:59:06.56114131 +0000 UTC m=+24.728737978"
Jul 15 04:59:07.382052 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:07.382025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:07.382293 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:07.382066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:07.382293 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:07.382066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:07.382293 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:07.382145    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:07.382293 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:07.382243    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:07.382516 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:07.382313    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:07.554073 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:07.553996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"4c2a01b44655a0dfe405d69521652cc43e386bb7cf4bcd960abcf055683d1ef2"}
Jul 15 04:59:07.558469 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:07.558435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl" event={"ID":"0f2506dd-dda6-410d-8881-397763b92799","Type":"ContainerStarted","Data":"c02738a42a9bb838ec84116d04ee93bbc0a37361fa19292925f6a78c16c9673a"}
Jul 15 04:59:07.577144 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:07.576933    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghjbl" podStartSLOduration=3.4570623019999998 podStartE2EDuration="24.576920123s" totalImagesPullingTime="21.119857821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:07.576413861 +0000 UTC m=+25.744010546" watchObservedRunningTime="2026-07-15 04:59:07.576920123 +0000 UTC m=+25.744516796"
Jul 15 04:59:09.382539 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.382511    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:09.382539 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.382527    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:09.382539 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.382530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:09.383132 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:09.382623    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:09.383132 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:09.382693    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:09.383132 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:09.382757    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:09.567515 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.567425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" event={"ID":"216bc077-7039-40ec-b23c-b3737183c581","Type":"ContainerStarted","Data":"aabd1563a28b72fdcc5512f878b887db6343ede7d303d82f3eaae518048afcf9"}
Jul 15 04:59:09.568045 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.567734    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:59:09.568045 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.567822    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:59:09.583444 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.583423    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:59:09.597423 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:09.597386    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd" podStartSLOduration=7.544127004 podStartE2EDuration="26.597372996s" totalImagesPullingTime="19.053245992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:09.597162678 +0000 UTC m=+27.764759349" watchObservedRunningTime="2026-07-15 04:59:09.597372996 +0000 UTC m=+27.764969667"
Jul 15 04:59:10.571639 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:10.571451    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d613161-cebc-4757-b35a-ddfb39c8501b" containerID="63263f7afe8c563ce4f5fd899d3481b9d04ed8b9028cee56894ea92269f750f2" exitCode=0
Jul 15 04:59:10.572477 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:10.571538    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerDied","Data":"63263f7afe8c563ce4f5fd899d3481b9d04ed8b9028cee56894ea92269f750f2"}
Jul 15 04:59:10.572477 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:10.572156    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:59:10.587370 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:10.587353    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:59:11.381546 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.381519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:11.381690 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.381625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:11.381690 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:11.381635    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:11.381690 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.381654    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:11.381797 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:11.381724    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:11.381797 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:11.381782    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:11.578992 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.576428    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d613161-cebc-4757-b35a-ddfb39c8501b" containerID="20c8d7d064b87dca423ca720e9f5d270afa59244e024f0e52566c03dd1f872b8" exitCode=0
Jul 15 04:59:11.578992 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.576615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerDied","Data":"20c8d7d064b87dca423ca720e9f5d270afa59244e024f0e52566c03dd1f872b8"}
Jul 15 04:59:11.728233 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.728193    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tq7nn"]
Jul 15 04:59:11.728404 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.728316    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:11.728452 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:11.728412    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:11.731675 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.731655    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-65s67"]
Jul 15 04:59:11.731798 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.731745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:11.731862 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:11.731816    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:11.732346 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.732327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nx4b2"]
Jul 15 04:59:11.732444 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:11.732404    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:11.732507 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:11.732486    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:12.581240 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:12.581073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerStarted","Data":"278879514fe2633fe758cb0b2155bf6d7c7feaf771652b5ecb387eac97210947"}
Jul 15 04:59:13.382330 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:13.382299    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:13.382548 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:13.382308    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:13.382548 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:13.382390    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:13.382548 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:13.382425    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:13.382548 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:13.382515    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:13.382791 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:13.382609    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:13.585630 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:13.585599    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d613161-cebc-4757-b35a-ddfb39c8501b" containerID="278879514fe2633fe758cb0b2155bf6d7c7feaf771652b5ecb387eac97210947" exitCode=0
Jul 15 04:59:13.586050 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:13.585638    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerDied","Data":"278879514fe2633fe758cb0b2155bf6d7c7feaf771652b5ecb387eac97210947"}
Jul 15 04:59:15.382188 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:15.382156    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:15.382774 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:15.382300    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:15.382774 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:15.382392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:15.382774 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:15.382505    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:15.382774 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:15.382556    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:15.382774 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:15.382627    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:16.000068 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:16.000031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:16.000237 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.000181    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:59:16.000290 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.000259    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret podName:6f13133e-0663-4a5a-b4d6-9a4e14609a76 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:48.000244505 +0000 UTC m=+66.167841167 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret") pod "global-pull-secret-syncer-tq7nn" (UID: "6f13133e-0663-4a5a-b4d6-9a4e14609a76") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:59:16.100479 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:16.100450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:16.100606 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:16.100487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:16.100606 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.100578    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:59:16.100606 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.100589    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:59:16.100606 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.100598    2581 projected.go:196] Error preparing data for projected volume kube-api-access-hf76r for pod openshift-network-diagnostics/network-check-target-65s67: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:59:16.100733 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.100633    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r podName:8d1616bb-ada0-4fdb-8b93-1ea20433aa69 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:48.10062198 +0000 UTC m=+66.268218632 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf76r" (UniqueName: "kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r") pod "network-check-target-65s67" (UID: "8d1616bb-ada0-4fdb-8b93-1ea20433aa69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:59:16.100733 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.100577    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:59:16.100733 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:16.100661    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:48.100654535 +0000 UTC m=+66.268251185 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:59:17.382197 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:17.382163    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:17.382197 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:17.382198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:17.382723 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:17.382221    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:17.382723 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:17.382305    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq7nn" podUID="6f13133e-0663-4a5a-b4d6-9a4e14609a76"
Jul 15 04:59:17.382723 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:17.382373    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-65s67" podUID="8d1616bb-ada0-4fdb-8b93-1ea20433aa69"
Jul 15 04:59:17.382723 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:17.382473    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 04:59:18.105686 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.105665    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeReady"
Jul 15 04:59:18.105786 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.105776    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 04:59:18.152570 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.152541    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8d24l"]
Jul 15 04:59:18.173157 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.173130    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k9765"]
Jul 15 04:59:18.173307 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.173285    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.175769 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.175745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 04:59:18.175881 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.175784    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 04:59:18.175881 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.175852    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crtzp\""
Jul 15 04:59:18.193402 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.193377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8d24l"]
Jul 15 04:59:18.193496 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.193410    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k9765"]
Jul 15 04:59:18.193496 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.193482    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.196140 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.196096    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-22jq7\""
Jul 15 04:59:18.196339 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.196145    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 04:59:18.196339 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.196158    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 04:59:18.196339 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.196158    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 04:59:18.316170 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.316146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6567160b-b7af-4b40-ad93-9e84b7338755-cert\") pod \"ingress-canary-k9765\" (UID: \"6567160b-b7af-4b40-ad93-9e84b7338755\") " pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.316279 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.316190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f07ac58a-7667-42a7-9f88-59801841c8f5-tmp-dir\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.316328 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.316264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf96p\" (UniqueName: \"kubernetes.io/projected/f07ac58a-7667-42a7-9f88-59801841c8f5-kube-api-access-rf96p\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.316328 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.316316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f07ac58a-7667-42a7-9f88-59801841c8f5-config-volume\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.316405 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.316334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.316453 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.316435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9krx\" (UniqueName: \"kubernetes.io/projected/6567160b-b7af-4b40-ad93-9e84b7338755-kube-api-access-b9krx\") pod \"ingress-canary-k9765\" (UID: \"6567160b-b7af-4b40-ad93-9e84b7338755\") " pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.416978 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.416924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9krx\" (UniqueName: \"kubernetes.io/projected/6567160b-b7af-4b40-ad93-9e84b7338755-kube-api-access-b9krx\") pod \"ingress-canary-k9765\" (UID: \"6567160b-b7af-4b40-ad93-9e84b7338755\") " pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.416978 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.416954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6567160b-b7af-4b40-ad93-9e84b7338755-cert\") pod \"ingress-canary-k9765\" (UID: \"6567160b-b7af-4b40-ad93-9e84b7338755\") " pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.417442 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.416983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f07ac58a-7667-42a7-9f88-59801841c8f5-tmp-dir\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.417442 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.417010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf96p\" (UniqueName: \"kubernetes.io/projected/f07ac58a-7667-42a7-9f88-59801841c8f5-kube-api-access-rf96p\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.417539 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.417129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f07ac58a-7667-42a7-9f88-59801841c8f5-config-volume\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.417584 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.417537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.417584 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.417546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f07ac58a-7667-42a7-9f88-59801841c8f5-tmp-dir\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.417691 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:18.417675    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:18.417879 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:18.417860    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:18.917835652 +0000 UTC m=+37.085432317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:18.420698 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.418839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f07ac58a-7667-42a7-9f88-59801841c8f5-config-volume\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.422351 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.422220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6567160b-b7af-4b40-ad93-9e84b7338755-cert\") pod \"ingress-canary-k9765\" (UID: \"6567160b-b7af-4b40-ad93-9e84b7338755\") " pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.425317 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.425297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf96p\" (UniqueName: \"kubernetes.io/projected/f07ac58a-7667-42a7-9f88-59801841c8f5-kube-api-access-rf96p\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.426192 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.426174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9krx\" (UniqueName: \"kubernetes.io/projected/6567160b-b7af-4b40-ad93-9e84b7338755-kube-api-access-b9krx\") pod \"ingress-canary-k9765\" (UID: \"6567160b-b7af-4b40-ad93-9e84b7338755\") " pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.502122 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.502096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k9765"
Jul 15 04:59:18.601542 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.601513    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d613161-cebc-4757-b35a-ddfb39c8501b" containerID="ae86fe7b10319e5e2b0e3adc02b18fc7580a3f4b3c57d2ea05f7e53c98616440" exitCode=0
Jul 15 04:59:18.601650 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.601553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerDied","Data":"ae86fe7b10319e5e2b0e3adc02b18fc7580a3f4b3c57d2ea05f7e53c98616440"}
Jul 15 04:59:18.627669 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.627643    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k9765"]
Jul 15 04:59:18.632849 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:59:18.632821    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6567160b_b7af_4b40_ad93_9e84b7338755.slice/crio-63c67923c7c39a8f48c1716afc35b5cfb7d55107aa16cfdda0d5794d588ba02e WatchSource:0}: Error finding container 63c67923c7c39a8f48c1716afc35b5cfb7d55107aa16cfdda0d5794d588ba02e: Status 404 returned error can't find the container with id 63c67923c7c39a8f48c1716afc35b5cfb7d55107aa16cfdda0d5794d588ba02e
Jul 15 04:59:18.920993 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:18.920927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:18.921114 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:18.921034    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:18.921114 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:18.921085    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:19.921069529 +0000 UTC m=+38.088666183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:19.381985 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.381952    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:19.382169 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.381995    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:19.382169 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.381952    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:19.385811 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.385769    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 04:59:19.385811 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.385782    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 04:59:19.386019 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.385833    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bt4r\""
Jul 15 04:59:19.386019 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.385843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 04:59:19.386019 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.385786    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lb4wr\""
Jul 15 04:59:19.386019 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.385914    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 04:59:19.609014 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.608977    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d613161-cebc-4757-b35a-ddfb39c8501b" containerID="444446965c411b93bfd9620176512da543115ed0e71e7ab1d355ea2d56a0eff5" exitCode=0
Jul 15 04:59:19.609460 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.609141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerDied","Data":"444446965c411b93bfd9620176512da543115ed0e71e7ab1d355ea2d56a0eff5"}
Jul 15 04:59:19.610906 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.610354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k9765" event={"ID":"6567160b-b7af-4b40-ad93-9e84b7338755","Type":"ContainerStarted","Data":"63c67923c7c39a8f48c1716afc35b5cfb7d55107aa16cfdda0d5794d588ba02e"}
Jul 15 04:59:19.928564 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:19.928240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:19.928698 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:19.928412    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:19.928698 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:19.928654    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:21.92862975 +0000 UTC m=+40.096226423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:20.616174 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:20.616144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" event={"ID":"0d613161-cebc-4757-b35a-ddfb39c8501b","Type":"ContainerStarted","Data":"be9300aa471e3bf0842a713284b0f88df1a61c53acb2b0c990c368bd03a19242"}
Jul 15 04:59:20.643100 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:20.643051    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2f9m2" podStartSLOduration=8.603167372 podStartE2EDuration="37.643034364s" totalImagesPullingTime="29.039866992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:20.642782746 +0000 UTC m=+38.810379419" watchObservedRunningTime="2026-07-15 04:59:20.643034364 +0000 UTC m=+38.810631038"
Jul 15 04:59:21.619948 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:21.619915    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k9765" event={"ID":"6567160b-b7af-4b40-ad93-9e84b7338755","Type":"ContainerStarted","Data":"e4aadda50fd02f03c598cad7ee924b56019fa72279989910bfa769820eccb521"}
Jul 15 04:59:21.635511 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:21.635468    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k9765" podStartSLOduration=1.6218483670000001 podStartE2EDuration="3.635453583s" totalImagesPullingTime="2.013605216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:21.634729379 +0000 UTC m=+39.802326051" watchObservedRunningTime="2026-07-15 04:59:21.635453583 +0000 UTC m=+39.803050236"
Jul 15 04:59:21.944619 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:21.944536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:21.944772 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:21.944639    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:21.944772 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:21.944688    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:25.944675086 +0000 UTC m=+44.112271736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:25.969006 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:25.968969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:25.969398 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:25.969115    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:25.969398 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:25.969177    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:33.96916232 +0000 UTC m=+52.136758970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:34.022846 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:34.022806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:34.023313 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:34.022956    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:34.023313 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:34.023024    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:50.023007682 +0000 UTC m=+68.190604332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:42.590807 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:42.590777    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llwcd"
Jul 15 04:59:48.009115 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.009068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:48.011780 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.011762    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 04:59:48.023495 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.023470    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f13133e-0663-4a5a-b4d6-9a4e14609a76-original-pull-secret\") pod \"global-pull-secret-syncer-tq7nn\" (UID: \"6f13133e-0663-4a5a-b4d6-9a4e14609a76\") " pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:48.109658 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.109623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 04:59:48.109783 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.109674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:48.112421 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.112405    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 04:59:48.112421 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.112415    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 04:59:48.120508 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:48.120492    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 04:59:48.120572 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:48.120560    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 05:00:52.120539 +0000 UTC m=+130.288135652 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : secret "metrics-daemon-secret" not found
Jul 15 04:59:48.122670 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.122657    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 04:59:48.133776 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.133757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf76r\" (UniqueName: \"kubernetes.io/projected/8d1616bb-ada0-4fdb-8b93-1ea20433aa69-kube-api-access-hf76r\") pod \"network-check-target-65s67\" (UID: \"8d1616bb-ada0-4fdb-8b93-1ea20433aa69\") " pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:48.196534 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.196515    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lb4wr\""
Jul 15 04:59:48.204742 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.204720    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:48.207022 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.207007    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq7nn"
Jul 15 04:59:48.342576 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.342521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-65s67"]
Jul 15 04:59:48.346390 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:59:48.346359    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1616bb_ada0_4fdb_8b93_1ea20433aa69.slice/crio-93a6dfbe9aaf1fb4f1dedabf9b7c60b83b02a4955ebcf6b424808a3483530a31 WatchSource:0}: Error finding container 93a6dfbe9aaf1fb4f1dedabf9b7c60b83b02a4955ebcf6b424808a3483530a31: Status 404 returned error can't find the container with id 93a6dfbe9aaf1fb4f1dedabf9b7c60b83b02a4955ebcf6b424808a3483530a31
Jul 15 04:59:48.354461 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.354440    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tq7nn"]
Jul 15 04:59:48.357296 ip-10-0-130-29 kubenswrapper[2581]: W0715 04:59:48.357276    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f13133e_0663_4a5a_b4d6_9a4e14609a76.slice/crio-dab31debe696c0cfdc0f142d195905fcb7a089c97e353513b18a5ca663443409 WatchSource:0}: Error finding container dab31debe696c0cfdc0f142d195905fcb7a089c97e353513b18a5ca663443409: Status 404 returned error can't find the container with id dab31debe696c0cfdc0f142d195905fcb7a089c97e353513b18a5ca663443409
Jul 15 04:59:48.677992 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.677965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tq7nn" event={"ID":"6f13133e-0663-4a5a-b4d6-9a4e14609a76","Type":"ContainerStarted","Data":"dab31debe696c0cfdc0f142d195905fcb7a089c97e353513b18a5ca663443409"}
Jul 15 04:59:48.678919 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:48.678900    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-65s67" event={"ID":"8d1616bb-ada0-4fdb-8b93-1ea20433aa69","Type":"ContainerStarted","Data":"93a6dfbe9aaf1fb4f1dedabf9b7c60b83b02a4955ebcf6b424808a3483530a31"}
Jul 15 04:59:50.024430 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:50.024391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 04:59:50.024864 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:50.024534    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:50.024864 ip-10-0-130-29 kubenswrapper[2581]: E0715 04:59:50.024612    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 05:00:22.024590625 +0000 UTC m=+100.192187286 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:53.694101 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:53.694061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tq7nn" event={"ID":"6f13133e-0663-4a5a-b4d6-9a4e14609a76","Type":"ContainerStarted","Data":"7aeb8f276f9f7db90947d220a273c0661585ccafd01f9db2f82c48c88fb1966e"}
Jul 15 04:59:53.695426 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:53.695405    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-65s67" event={"ID":"8d1616bb-ada0-4fdb-8b93-1ea20433aa69","Type":"ContainerStarted","Data":"72fd2df5bd88d3b07573f5e5a785ea502a8b092d1b287696b86a8019b39a5466"}
Jul 15 04:59:53.695546 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:53.695528    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 04:59:53.711128 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:53.711088    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tq7nn" podStartSLOduration=65.988488188 podStartE2EDuration="1m10.711076983s" totalImagesPullingTime="4.722588795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:53.710703561 +0000 UTC m=+71.878300224" watchObservedRunningTime="2026-07-15 04:59:53.711076983 +0000 UTC m=+71.878673654"
Jul 15 04:59:53.724775 ip-10-0-130-29 kubenswrapper[2581]: I0715 04:59:53.724735    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-65s67" podStartSLOduration=66.002286646 podStartE2EDuration="1m10.724719979s" totalImagesPullingTime="4.722433333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:53.724513659 +0000 UTC m=+71.892110327" watchObservedRunningTime="2026-07-15 04:59:53.724719979 +0000 UTC m=+71.892316651"
Jul 15 05:00:13.932275 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.932148    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"]
Jul 15 05:00:13.934831 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.934813    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:13.937620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.937602    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 05:00:13.937748 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.937644    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 05:00:13.938622 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.938604    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 05:00:13.938746 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.938637    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h4qpw\""
Jul 15 05:00:13.938810 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.938776    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 15 05:00:13.940386 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.940365    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"]
Jul 15 05:00:13.942591 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.942577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:13.945676 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.945660    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 15 05:00:13.945676 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.945670    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 15 05:00:13.945915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.945901    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 15 05:00:13.946415 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.946394    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"]
Jul 15 05:00:13.946691 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.946678    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 15 05:00:13.955847 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.955827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"]
Jul 15 05:00:13.978571 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:13.978652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/642fc823-a947-4088-baf2-5303557ab537-additional-service-ca\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:13.978652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-ca\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:13.978652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz\" (UID: \"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:13.978652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bllfz\" (UniqueName: \"kubernetes.io/projected/642fc823-a947-4088-baf2-5303557ab537-kube-api-access-bllfz\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:13.978791 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs62f\" (UniqueName: \"kubernetes.io/projected/c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b-kube-api-access-xs62f\") pod \"managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz\" (UID: \"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:13.978791 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-hub\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:13.978791 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:13.978783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.079585 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.079701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/642fc823-a947-4088-baf2-5303557ab537-additional-service-ca\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.079701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-ca\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.079701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz\" (UID: \"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:14.079701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bllfz\" (UniqueName: \"kubernetes.io/projected/642fc823-a947-4088-baf2-5303557ab537-kube-api-access-bllfz\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.079905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs62f\" (UniqueName: \"kubernetes.io/projected/c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b-kube-api-access-xs62f\") pod \"managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz\" (UID: \"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:14.079905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-hub\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.079905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.079805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.080721 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.080689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/642fc823-a947-4088-baf2-5303557ab537-additional-service-ca\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.082126 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.082080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.082320 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.082299    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz\" (UID: \"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:14.082404 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.082337    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.082487 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.082469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-ca\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.082533 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.082494    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/642fc823-a947-4088-baf2-5303557ab537-hub\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.087347 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.087326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bllfz\" (UniqueName: \"kubernetes.io/projected/642fc823-a947-4088-baf2-5303557ab537-kube-api-access-bllfz\") pod \"cluster-proxy-proxy-agent-797f6756f8-5qw6s\" (UID: \"642fc823-a947-4088-baf2-5303557ab537\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.087653 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.087629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs62f\" (UniqueName: \"kubernetes.io/projected/c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b-kube-api-access-xs62f\") pod \"managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz\" (UID: \"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:14.257065 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.257010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"
Jul 15 05:00:14.264536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.264514    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:00:14.380804 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.380774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz"]
Jul 15 05:00:14.383914 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:00:14.383885    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc72b8c16_9f66_4f44_af71_8d9a5dd0fd1b.slice/crio-178b991c2d80193fd619b7e23a381f6993da039a250d162f9b8e30b5426f0d7c WatchSource:0}: Error finding container 178b991c2d80193fd619b7e23a381f6993da039a250d162f9b8e30b5426f0d7c: Status 404 returned error can't find the container with id 178b991c2d80193fd619b7e23a381f6993da039a250d162f9b8e30b5426f0d7c
Jul 15 05:00:14.399419 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.399391    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"]
Jul 15 05:00:14.401996 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:00:14.401973    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod642fc823_a947_4088_baf2_5303557ab537.slice/crio-1cd4359e8bd5a819a1aeb95aac7e51a2fb819f4a49898534598526106612388d WatchSource:0}: Error finding container 1cd4359e8bd5a819a1aeb95aac7e51a2fb819f4a49898534598526106612388d: Status 404 returned error can't find the container with id 1cd4359e8bd5a819a1aeb95aac7e51a2fb819f4a49898534598526106612388d
Jul 15 05:00:14.745495 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.745460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" event={"ID":"642fc823-a947-4088-baf2-5303557ab537","Type":"ContainerStarted","Data":"1cd4359e8bd5a819a1aeb95aac7e51a2fb819f4a49898534598526106612388d"}
Jul 15 05:00:14.746518 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:14.746496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz" event={"ID":"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b","Type":"ContainerStarted","Data":"178b991c2d80193fd619b7e23a381f6993da039a250d162f9b8e30b5426f0d7c"}
Jul 15 05:00:18.761071 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:18.761030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" event={"ID":"642fc823-a947-4088-baf2-5303557ab537","Type":"ContainerStarted","Data":"de84a498fb0d9e1cf5ba15289c9f14e28b9283448ab675572f3aa7e687af12ee"}
Jul 15 05:00:18.761071 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:18.761077    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" event={"ID":"642fc823-a947-4088-baf2-5303557ab537","Type":"ContainerStarted","Data":"e3749bb4acda6d952f34fcb90fba1182670ade63384c33d716cb8a204e13e3dd"}
Jul 15 05:00:18.761625 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:18.761095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" event={"ID":"642fc823-a947-4088-baf2-5303557ab537","Type":"ContainerStarted","Data":"48682d0b0ad13a50e14704feeb84db893fb05afdc409cd4d1b691c7e9c098d33"}
Jul 15 05:00:18.762409 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:18.762389    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz" event={"ID":"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b","Type":"ContainerStarted","Data":"68c273c82474edecf6f2c1165229c4cfb0b5dfbbb98fdaecd00d7cf0cca4502c"}
Jul 15 05:00:18.780612 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:18.780570    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" podStartSLOduration=2.000762935 podStartE2EDuration="5.780558047s" totalImagesPullingTime="3.779795112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:00:18.779014171 +0000 UTC m=+96.946610858" watchObservedRunningTime="2026-07-15 05:00:18.780558047 +0000 UTC m=+96.948154742"
Jul 15 05:00:18.793423 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:18.793385    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz" podStartSLOduration=2.005934022 podStartE2EDuration="5.793373192s" totalImagesPullingTime="3.78743917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:00:18.793126627 +0000 UTC m=+96.960723300" watchObservedRunningTime="2026-07-15 05:00:18.793373192 +0000 UTC m=+96.960969864"
Jul 15 05:00:22.040443 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:22.040394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 05:00:22.040835 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:00:22.040524    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 05:00:22.040835 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:00:22.040576    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 05:01:26.040563788 +0000 UTC m=+164.208160437 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 05:00:24.700597 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:24.700567    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-65s67"
Jul 15 05:00:52.150811 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:00:52.150769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 05:00:52.151326 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:00:52.150922    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 05:00:52.151326 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:00:52.150997    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 05:02:54.150981697 +0000 UTC m=+252.318578347 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : secret "metrics-daemon-secret" not found
Jul 15 05:01:21.183055 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:01:21.183008    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8d24l" podUID="f07ac58a-7667-42a7-9f88-59801841c8f5"
Jul 15 05:01:21.941174 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:01:21.941148    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8d24l"
Jul 15 05:01:22.400897 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:01:22.400866    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 05:01:26.069083 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:01:26.069041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 05:01:26.069461 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:01:26.069166    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 05:01:26.069461 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:01:26.069246    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls podName:f07ac58a-7667-42a7-9f88-59801841c8f5 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:28.069229876 +0000 UTC m=+286.236826527 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls") pod "dns-default-8d24l" (UID: "f07ac58a-7667-42a7-9f88-59801841c8f5") : secret "dns-default-metrics-tls" not found
Jul 15 05:01:37.382532 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:01:37.382460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 05:02:19.099520 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:02:19.099486    2581 generic.go:356] "Generic (PLEG): container finished" podID="c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b" containerID="68c273c82474edecf6f2c1165229c4cfb0b5dfbbb98fdaecd00d7cf0cca4502c" exitCode=255
Jul 15 05:02:19.099921 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:02:19.099533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz" event={"ID":"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b","Type":"ContainerDied","Data":"68c273c82474edecf6f2c1165229c4cfb0b5dfbbb98fdaecd00d7cf0cca4502c"}
Jul 15 05:02:19.099921 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:02:19.099781    2581 scope.go:117] "RemoveContainer" containerID="68c273c82474edecf6f2c1165229c4cfb0b5dfbbb98fdaecd00d7cf0cca4502c"
Jul 15 05:02:20.103263 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:02:20.103231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6db6d8f79d-nw4qz" event={"ID":"c72b8c16-9f66-4f44-af71-8d9a5dd0fd1b","Type":"ContainerStarted","Data":"a0a6e0d06bad9566d8c6cb149bc1607205e6b8062f6a65f4207e7eb923f4b8a7"}
Jul 15 05:02:54.238646 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:02:54.238610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 05:02:54.239141 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:02:54.238764    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 05:02:54.239141 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:02:54.238828    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs podName:971ca253-982f-4e83-86ad-cba13a9970e0 nodeName:}" failed. No retries permitted until 2026-07-15 05:04:56.238811224 +0000 UTC m=+374.406407874 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs") pod "network-metrics-daemon-nx4b2" (UID: "971ca253-982f-4e83-86ad-cba13a9970e0") : secret "metrics-daemon-secret" not found
Jul 15 05:03:08.082919 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.082839    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-knln7"]
Jul 15 05:03:08.085029 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.085010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.088460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.088432    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7zgl9\""
Jul 15 05:03:08.089562 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.089542    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 05:03:08.089684 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.089583    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 05:03:08.089684 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.089591    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 05:03:08.089684 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.089610    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 05:03:08.100380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.100348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-knln7"]
Jul 15 05:03:08.233839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.233806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3550e420-fe88-4fb6-aee3-3393f8e2ee21-crio-socket\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.233839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.233842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.234049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.233911    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.234049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.233934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d75z\" (UniqueName: \"kubernetes.io/projected/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-api-access-4d75z\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.234049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.233957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3550e420-fe88-4fb6-aee3-3393f8e2ee21-data-volume\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.335365 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.335267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3550e420-fe88-4fb6-aee3-3393f8e2ee21-crio-socket\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.335365 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.335321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.335365 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.335359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.335565 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.335377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d75z\" (UniqueName: \"kubernetes.io/projected/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-api-access-4d75z\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.335565 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.335396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3550e420-fe88-4fb6-aee3-3393f8e2ee21-data-volume\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.335565 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.335395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3550e420-fe88-4fb6-aee3-3393f8e2ee21-crio-socket\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.335565 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.335465    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:08.335565 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.335503    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:08.335565 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.335532    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:08.835516425 +0000 UTC m=+267.003113075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:08.335565 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.335561    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:08.835545684 +0000 UTC m=+267.003142339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:08.335780 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.335656    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3550e420-fe88-4fb6-aee3-3393f8e2ee21-data-volume\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.346037 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.346017    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d75z\" (UniqueName: \"kubernetes.io/projected/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-api-access-4d75z\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.838883 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.838853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.839033 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:08.838907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:08.839033 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.839008    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:08.839102 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.839066    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:09.839052024 +0000 UTC m=+268.006648677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:08.839102 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.839008    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:08.839174 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:08.839149    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:09.839134919 +0000 UTC m=+268.006731571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:09.846770 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:09.846732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:09.847142 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:09.846789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:09.847142 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:09.846853    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:09.847142 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:09.846875    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:09.847142 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:09.846904    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:11.84689067 +0000 UTC m=+270.014487320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:09.847142 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:09.846926    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:11.846914326 +0000 UTC m=+270.014510983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:10.881579 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:10.881549    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7ndr7_f9d9807f-4795-4f9c-8ceb-e41e98fdfed0/dns-node-resolver/0.log"
Jul 15 05:03:11.864295 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:11.864245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:11.864480 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:11.864327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:11.864480 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:11.864368    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:11.864480 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:11.864416    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:11.864480 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:11.864438    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:15.864421762 +0000 UTC m=+274.032018412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:11.864480 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:11.864458    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:15.864445197 +0000 UTC m=+274.032041850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:12.081460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:12.081434    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n2nlx_321e957c-8792-4fa3-974d-e9352212e364/node-ca/0.log"
Jul 15 05:03:12.681847 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:12.681818    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k9765_6567160b-b7af-4b40-ad93-9e84b7338755/serve-healthcheck-canary/0.log"
Jul 15 05:03:15.894474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:15.894437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:15.894927 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:15.894493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:15.894927 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:15.894566    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:15.894927 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:15.894574    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:15.894927 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:15.894617    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:23.894602595 +0000 UTC m=+282.062199246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:15.894927 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:15.894629    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls podName:3550e420-fe88-4fb6-aee3-3393f8e2ee21 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:23.894623522 +0000 UTC m=+282.062220171 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knln7" (UID: "3550e420-fe88-4fb6-aee3-3393f8e2ee21") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:23.963665 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:23.963611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:23.964157 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:23.963687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:23.964348 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:23.964328    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3550e420-fe88-4fb6-aee3-3393f8e2ee21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:23.965973 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:23.965953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3550e420-fe88-4fb6-aee3-3393f8e2ee21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knln7\" (UID: \"3550e420-fe88-4fb6-aee3-3393f8e2ee21\") " pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:23.995009 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:23.994970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-knln7"
Jul 15 05:03:24.118739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:24.118705    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-knln7"]
Jul 15 05:03:24.121919 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:03:24.121893    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3550e420_fe88_4fb6_aee3_3393f8e2ee21.slice/crio-fb2e224d550b4bc75237ff5fa3336d1059dfea44d30a7fcb51010c5530a168cf WatchSource:0}: Error finding container fb2e224d550b4bc75237ff5fa3336d1059dfea44d30a7fcb51010c5530a168cf: Status 404 returned error can't find the container with id fb2e224d550b4bc75237ff5fa3336d1059dfea44d30a7fcb51010c5530a168cf
Jul 15 05:03:24.275596 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:24.275515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knln7" event={"ID":"3550e420-fe88-4fb6-aee3-3393f8e2ee21","Type":"ContainerStarted","Data":"fb2e224d550b4bc75237ff5fa3336d1059dfea44d30a7fcb51010c5530a168cf"}
Jul 15 05:03:24.942045 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:24.941989    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8d24l" podUID="f07ac58a-7667-42a7-9f88-59801841c8f5"
Jul 15 05:03:25.278962 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:25.278885    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8d24l"
Jul 15 05:03:28.094172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:28.094137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 05:03:28.096432 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:28.096411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f07ac58a-7667-42a7-9f88-59801841c8f5-metrics-tls\") pod \"dns-default-8d24l\" (UID: \"f07ac58a-7667-42a7-9f88-59801841c8f5\") " pod="openshift-dns/dns-default-8d24l"
Jul 15 05:03:28.282073 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:28.282043    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crtzp\""
Jul 15 05:03:28.290426 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:28.290389    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8d24l"
Jul 15 05:03:28.292052 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:28.292023    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knln7" event={"ID":"3550e420-fe88-4fb6-aee3-3393f8e2ee21","Type":"ContainerStarted","Data":"fac24acd5a3054b5351391ca6100076bafb8988e412ebadcdb1bfb9d0849060b"}
Jul 15 05:03:28.481142 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:28.481101    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8d24l"]
Jul 15 05:03:28.483981 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:03:28.483956    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07ac58a_7667_42a7_9f88_59801841c8f5.slice/crio-accb8b1a629c1bcb1c102fa0155b8a0aa1ca8cc25d16b008b87386acba4e9a7f WatchSource:0}: Error finding container accb8b1a629c1bcb1c102fa0155b8a0aa1ca8cc25d16b008b87386acba4e9a7f: Status 404 returned error can't find the container with id accb8b1a629c1bcb1c102fa0155b8a0aa1ca8cc25d16b008b87386acba4e9a7f
Jul 15 05:03:29.298483 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:29.298427    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knln7" event={"ID":"3550e420-fe88-4fb6-aee3-3393f8e2ee21","Type":"ContainerStarted","Data":"0905ea40edced6bc28f239ebe1660d5275e01949cc6b9e6ae13b8e2ff4a19bb8"}
Jul 15 05:03:29.299908 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:29.299867    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8d24l" event={"ID":"f07ac58a-7667-42a7-9f88-59801841c8f5","Type":"ContainerStarted","Data":"accb8b1a629c1bcb1c102fa0155b8a0aa1ca8cc25d16b008b87386acba4e9a7f"}
Jul 15 05:03:31.308760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:31.308657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8d24l" event={"ID":"f07ac58a-7667-42a7-9f88-59801841c8f5","Type":"ContainerStarted","Data":"dee9a37f20dee564c872b7dcdfe85fc63f58fd180bf94386b88db30aad0ec871"}
Jul 15 05:03:31.312545 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:31.312496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knln7" event={"ID":"3550e420-fe88-4fb6-aee3-3393f8e2ee21","Type":"ContainerStarted","Data":"80482055f7ae8dddba66f408e78de4d18bd02a6ca63f595729e2f953414225c3"}
Jul 15 05:03:31.337835 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:31.337785    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-knln7" podStartSLOduration=16.389073746 podStartE2EDuration="23.337768346s" totalImagesPullingTime="6.9486946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 05:03:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:31.33728598 +0000 UTC m=+289.504882654" watchObservedRunningTime="2026-07-15 05:03:31.337768346 +0000 UTC m=+289.505365018"
Jul 15 05:03:32.316917 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:32.316882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8d24l" event={"ID":"f07ac58a-7667-42a7-9f88-59801841c8f5","Type":"ContainerStarted","Data":"b3d873daafad41e8060d1c9b2e5d9dfb5f824380278a49894628f640cd5f6efe"}
Jul 15 05:03:32.317321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:32.317108    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8d24l"
Jul 15 05:03:32.334734 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:32.334669    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8d24l" podStartSLOduration=251.621033948 podStartE2EDuration="4m14.334655065s" totalImagesPullingTime="2.713621117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:32.334130951 +0000 UTC m=+290.501727620" watchObservedRunningTime="2026-07-15 05:03:32.334655065 +0000 UTC m=+290.502251738"
Jul 15 05:03:40.383327 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:40.383288    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nx4b2" podUID="971ca253-982f-4e83-86ad-cba13a9970e0"
Jul 15 05:03:42.239790 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:42.239761    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 05:03:42.321785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:42.321754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8d24l"
Jul 15 05:03:44.266024 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:44.265980    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" podUID="642fc823-a947-4088-baf2-5303557ab537" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 15 05:03:45.752516 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.752484    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77887fdb84-nrd58"]
Jul 15 05:03:45.757299 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.757279    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.760258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.760227    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 05:03:45.760258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.760244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 05:03:45.760447 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.760339    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 05:03:45.761566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.761548    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 05:03:45.761682 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.761594    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5pgb9\""
Jul 15 05:03:45.761682 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.761617    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 05:03:45.761682 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.761651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 05:03:45.761927 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.761912    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 05:03:45.765985 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.765961    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 05:03:45.767130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.767110    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77887fdb84-nrd58"]
Jul 15 05:03:45.820303 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.820260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-serving-cert\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.820471 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.820316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-oauth-serving-cert\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.820471 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.820356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-oauth-config\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.820471 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.820377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-trusted-ca-bundle\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.820471 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.820418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-console-config\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.820620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.820471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-service-ca\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.820620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.820492    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zflpj\" (UniqueName: \"kubernetes.io/projected/b6d8af17-2220-4b6e-ae61-d38236edd083-kube-api-access-zflpj\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.921291 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.921255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-service-ca\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.921291 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.921291    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zflpj\" (UniqueName: \"kubernetes.io/projected/b6d8af17-2220-4b6e-ae61-d38236edd083-kube-api-access-zflpj\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.921519 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.921309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-serving-cert\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.921519 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.921333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-oauth-serving-cert\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.921519 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.921376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-oauth-config\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.921519 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.921411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-trusted-ca-bundle\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.921519 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.921462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-console-config\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.922132 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.922105    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-service-ca\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.922284 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.922147    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-console-config\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.922284 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.922147    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-oauth-serving-cert\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.922284 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.922238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-trusted-ca-bundle\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.923802 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.923784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-oauth-config\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.923900 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.923887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-serving-cert\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:45.930000 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:45.929979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zflpj\" (UniqueName: \"kubernetes.io/projected/b6d8af17-2220-4b6e-ae61-d38236edd083-kube-api-access-zflpj\") pod \"console-77887fdb84-nrd58\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") " pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:46.066198 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:46.066157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:46.183707 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:46.183671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77887fdb84-nrd58"]
Jul 15 05:03:46.186249 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:03:46.186220    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d8af17_2220_4b6e_ae61_d38236edd083.slice/crio-2cfe982ae55c57564d4033991de813e41694bc93331ef8fe8fce9adb0bac597b WatchSource:0}: Error finding container 2cfe982ae55c57564d4033991de813e41694bc93331ef8fe8fce9adb0bac597b: Status 404 returned error can't find the container with id 2cfe982ae55c57564d4033991de813e41694bc93331ef8fe8fce9adb0bac597b
Jul 15 05:03:46.188286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:46.188272    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:03:46.357441 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:46.357363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77887fdb84-nrd58" event={"ID":"b6d8af17-2220-4b6e-ae61-d38236edd083","Type":"ContainerStarted","Data":"2cfe982ae55c57564d4033991de813e41694bc93331ef8fe8fce9adb0bac597b"}
Jul 15 05:03:49.368365 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:49.368322    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77887fdb84-nrd58" event={"ID":"b6d8af17-2220-4b6e-ae61-d38236edd083","Type":"ContainerStarted","Data":"3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed"}
Jul 15 05:03:49.392244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:49.392174    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77887fdb84-nrd58" podStartSLOduration=1.737012311 podStartE2EDuration="4.39216021s" totalImagesPullingTime="2.655147899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:49.391577897 +0000 UTC m=+307.559174569" watchObservedRunningTime="2026-07-15 05:03:49.39216021 +0000 UTC m=+307.559756882"
Jul 15 05:03:53.381903 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:53.381876    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 05:03:54.265766 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:54.265730    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" podUID="642fc823-a947-4088-baf2-5303557ab537" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 15 05:03:56.066570 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.066535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:56.067144 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.067121    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:56.067821 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.067800    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"]
Jul 15 05:03:56.087739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.087712    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"]
Jul 15 05:03:56.087860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.087787    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:56.087860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.087827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.090488 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.090461    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 15 05:03:56.090614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.090595    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 05:03:56.090686 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.090676    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 15 05:03:56.090781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.090763    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 05:03:56.091615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.091598    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 05:03:56.091615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.091604    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mmx5b\""
Jul 15 05:03:56.101060 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.101043    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cpxwp"]
Jul 15 05:03:56.103082 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.103068    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.105653 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.105630    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fl7xt\""
Jul 15 05:03:56.105780 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.105764    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 05:03:56.106007 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.105991    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 05:03:56.106192 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.106180    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 05:03:56.191911 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.191887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-tls\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192025 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.191939    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-root\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192138 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-accelerators-collector-config\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192189 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.192258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-sys\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192303 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrqn\" (UniqueName: \"kubernetes.io/projected/8995cb25-9a05-4459-a5f9-5a684d8addda-kube-api-access-jqrqn\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192360 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vm8n\" (UniqueName: \"kubernetes.io/projected/635175b9-d7ac-4ea2-aef1-d6e716193e8c-kube-api-access-9vm8n\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.192421 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192462 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-textfile\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192514 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.192566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-wtmp\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.192618 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/635175b9-d7ac-4ea2-aef1-d6e716193e8c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.192618 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.192608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8995cb25-9a05-4459-a5f9-5a684d8addda-metrics-client-ca\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293319 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-wtmp\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293440 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/635175b9-d7ac-4ea2-aef1-d6e716193e8c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.293440 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8995cb25-9a05-4459-a5f9-5a684d8addda-metrics-client-ca\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293440 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-tls\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293440 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293438    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-root\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-wtmp\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-accelerators-collector-config\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-root\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.293643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293539    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-sys\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrqn\" (UniqueName: \"kubernetes.io/projected/8995cb25-9a05-4459-a5f9-5a684d8addda-kube-api-access-jqrqn\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.293643 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:56.293539    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293656    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8995cb25-9a05-4459-a5f9-5a684d8addda-sys\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:56.293677    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-tls podName:8995cb25-9a05-4459-a5f9-5a684d8addda nodeName:}" failed. No retries permitted until 2026-07-15 05:03:56.793653072 +0000 UTC m=+314.961249727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-tls") pod "node-exporter-cpxwp" (UID: "8995cb25-9a05-4459-a5f9-5a684d8addda") : secret "node-exporter-tls" not found
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vm8n\" (UniqueName: \"kubernetes.io/projected/635175b9-d7ac-4ea2-aef1-d6e716193e8c-kube-api-access-9vm8n\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-textfile\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:56.293880    2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:03:56.293918    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-tls podName:635175b9-d7ac-4ea2-aef1-d6e716193e8c nodeName:}" failed. No retries permitted until 2026-07-15 05:03:56.793905167 +0000 UTC m=+314.961501818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-hx687" (UID: "635175b9-d7ac-4ea2-aef1-d6e716193e8c") : secret "openshift-state-metrics-tls" not found
Jul 15 05:03:56.294000 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.293976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8995cb25-9a05-4459-a5f9-5a684d8addda-metrics-client-ca\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.294506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.294021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-accelerators-collector-config\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.294506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.294042    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-textfile\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.294506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.294361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/635175b9-d7ac-4ea2-aef1-d6e716193e8c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.295884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.295862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.295998 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.295979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.302472 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.302448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrqn\" (UniqueName: \"kubernetes.io/projected/8995cb25-9a05-4459-a5f9-5a684d8addda-kube-api-access-jqrqn\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.303162 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.303141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vm8n\" (UniqueName: \"kubernetes.io/projected/635175b9-d7ac-4ea2-aef1-d6e716193e8c-kube-api-access-9vm8n\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.399582 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.399525    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:03:56.797019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.796935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.797019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.796982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-tls\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.799390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.799360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8995cb25-9a05-4459-a5f9-5a684d8addda-node-exporter-tls\") pod \"node-exporter-cpxwp\" (UID: \"8995cb25-9a05-4459-a5f9-5a684d8addda\") " pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:56.799506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.799467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/635175b9-d7ac-4ea2-aef1-d6e716193e8c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hx687\" (UID: \"635175b9-d7ac-4ea2-aef1-d6e716193e8c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:56.996850 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:56.996823    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"
Jul 15 05:03:57.010883 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:57.010860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cpxwp"
Jul 15 05:03:57.019686 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:03:57.019667    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8995cb25_9a05_4459_a5f9_5a684d8addda.slice/crio-e189aa9533c3ec8b729e13a83a5952ddff0cc1b6833be4c2ae7f3409a255807d WatchSource:0}: Error finding container e189aa9533c3ec8b729e13a83a5952ddff0cc1b6833be4c2ae7f3409a255807d: Status 404 returned error can't find the container with id e189aa9533c3ec8b729e13a83a5952ddff0cc1b6833be4c2ae7f3409a255807d
Jul 15 05:03:57.124019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:57.123975    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hx687"]
Jul 15 05:03:57.126499 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:03:57.126478    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635175b9_d7ac_4ea2_aef1_d6e716193e8c.slice/crio-5d01b3a6e0ad0bac047b20aeecb885ecf714c4a71ee95ac99e54417429961e17 WatchSource:0}: Error finding container 5d01b3a6e0ad0bac047b20aeecb885ecf714c4a71ee95ac99e54417429961e17: Status 404 returned error can't find the container with id 5d01b3a6e0ad0bac047b20aeecb885ecf714c4a71ee95ac99e54417429961e17
Jul 15 05:03:57.400396 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:57.400311    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cpxwp" event={"ID":"8995cb25-9a05-4459-a5f9-5a684d8addda","Type":"ContainerStarted","Data":"e189aa9533c3ec8b729e13a83a5952ddff0cc1b6833be4c2ae7f3409a255807d"}
Jul 15 05:03:57.402281 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:57.402254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687" event={"ID":"635175b9-d7ac-4ea2-aef1-d6e716193e8c","Type":"ContainerStarted","Data":"7994a22ebac5899c994e3698a17b6179ceef4387a0550eb9942ef85a53385fe1"}
Jul 15 05:03:57.402390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:57.402304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687" event={"ID":"635175b9-d7ac-4ea2-aef1-d6e716193e8c","Type":"ContainerStarted","Data":"455bfd81f2dc012f78f3d0a74de9b5fd647887a65a44f3496743ef035909302b"}
Jul 15 05:03:57.402390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:57.402327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687" event={"ID":"635175b9-d7ac-4ea2-aef1-d6e716193e8c","Type":"ContainerStarted","Data":"5d01b3a6e0ad0bac047b20aeecb885ecf714c4a71ee95ac99e54417429961e17"}
Jul 15 05:03:58.149543 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.149464    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"]
Jul 15 05:03:58.152606 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.152589    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.155332 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.155306    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 05:03:58.155457 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.155338    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 05:03:58.155457 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.155372    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 05:03:58.155578 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.155556    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dmep2omfp0kav\""
Jul 15 05:03:58.155701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.155588    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 05:03:58.155701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.155592    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 05:03:58.155701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.155650    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q2t5d\""
Jul 15 05:03:58.166042 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.166019    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"]
Jul 15 05:03:58.208964 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.208935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.209119 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.208978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-grpc-tls\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.209119 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.209010    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.209119 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.209098    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac839800-6548-4332-98d0-0c5220844e48-metrics-client-ca\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.209307 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.209143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.209307 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.209197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7j6p\" (UniqueName: \"kubernetes.io/projected/ac839800-6548-4332-98d0-0c5220844e48-kube-api-access-z7j6p\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.209408 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.209305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-tls\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.209408 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.209376    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.309916 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.309876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.309933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac839800-6548-4332-98d0-0c5220844e48-metrics-client-ca\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.309963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.309991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7j6p\" (UniqueName: \"kubernetes.io/projected/ac839800-6548-4332-98d0-0c5220844e48-kube-api-access-z7j6p\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.310015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-tls\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.310063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.310113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310463 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.310383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-grpc-tls\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.310868 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.310815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac839800-6548-4332-98d0-0c5220844e48-metrics-client-ca\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.312668 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.312640    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.312767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.312645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.313091 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.313069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.313197 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.313177    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-grpc-tls\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.313267 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.313244    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.313301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.313272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ac839800-6548-4332-98d0-0c5220844e48-secret-thanos-querier-tls\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.317384 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.317365    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7j6p\" (UniqueName: \"kubernetes.io/projected/ac839800-6548-4332-98d0-0c5220844e48-kube-api-access-z7j6p\") pod \"thanos-querier-5bdcbcd669-ssmps\" (UID: \"ac839800-6548-4332-98d0-0c5220844e48\") " pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.406638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.406552    2581 generic.go:356] "Generic (PLEG): container finished" podID="8995cb25-9a05-4459-a5f9-5a684d8addda" containerID="dd3aca815ef7face80884a74b6fd9e0b625845fb742a7b72b1b69fe85f034ed9" exitCode=0
Jul 15 05:03:58.406638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.406618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cpxwp" event={"ID":"8995cb25-9a05-4459-a5f9-5a684d8addda","Type":"ContainerDied","Data":"dd3aca815ef7face80884a74b6fd9e0b625845fb742a7b72b1b69fe85f034ed9"}
Jul 15 05:03:58.461713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.461693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:03:58.606303 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:58.606273    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"]
Jul 15 05:03:58.767009 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:03:58.766938    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac839800_6548_4332_98d0_0c5220844e48.slice/crio-183164135b7d413fae6d141f4b9b3c7e7771c6c0bcf5d72e7f66ef9adcfc6324 WatchSource:0}: Error finding container 183164135b7d413fae6d141f4b9b3c7e7771c6c0bcf5d72e7f66ef9adcfc6324: Status 404 returned error can't find the container with id 183164135b7d413fae6d141f4b9b3c7e7771c6c0bcf5d72e7f66ef9adcfc6324
Jul 15 05:03:59.411302 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:59.411271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" event={"ID":"ac839800-6548-4332-98d0-0c5220844e48","Type":"ContainerStarted","Data":"183164135b7d413fae6d141f4b9b3c7e7771c6c0bcf5d72e7f66ef9adcfc6324"}
Jul 15 05:03:59.413172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:59.413148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cpxwp" event={"ID":"8995cb25-9a05-4459-a5f9-5a684d8addda","Type":"ContainerStarted","Data":"db01b822ab4677f00517f0a1e86bd1e08a0302d848d5ee2b52a58af7d7b83000"}
Jul 15 05:03:59.413301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:59.413188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cpxwp" event={"ID":"8995cb25-9a05-4459-a5f9-5a684d8addda","Type":"ContainerStarted","Data":"1ec71c6390e09a30871b7e2587e826c8ae69e86ae82c97c3699571b6a67fbfc9"}
Jul 15 05:03:59.415160 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:59.415138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687" event={"ID":"635175b9-d7ac-4ea2-aef1-d6e716193e8c","Type":"ContainerStarted","Data":"837d985ea2b1829da2b8828c9bb55a7b7b1b6998a9e6e61408da01635d43489d"}
Jul 15 05:03:59.432379 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:59.432340    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cpxwp" podStartSLOduration=2.5629804849999998 podStartE2EDuration="3.432327058s" totalImagesPullingTime="804.536395ms" totalInitContainerRuntime="64.810178ms" isStatefulPod=true podCreationTimestamp="2026-07-15 05:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:59.431082635 +0000 UTC m=+317.598679323" watchObservedRunningTime="2026-07-15 05:03:59.432327058 +0000 UTC m=+317.599923730"
Jul 15 05:03:59.448549 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:03:59.448514    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hx687" podStartSLOduration=1.890216776 podStartE2EDuration="3.44850185s" totalImagesPullingTime="1.558285074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:59.447726339 +0000 UTC m=+317.615323010" watchObservedRunningTime="2026-07-15 05:03:59.44850185 +0000 UTC m=+317.616098522"
Jul 15 05:04:00.487233 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.487188    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"]
Jul 15 05:04:00.489414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.489400    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.492612 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.492593    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 15 05:04:00.492853 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.492838    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 15 05:04:00.493066 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.493052    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6kb8j\""
Jul 15 05:04:00.499703 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.499687    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 05:04:00.500226 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.500195    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 15 05:04:00.500351 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.500336    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cgdrngmb2a60m\""
Jul 15 05:04:00.514001 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.512771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"]
Jul 15 05:04:00.529225 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.529188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-secret-metrics-server-client-certs\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.529325 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.529237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ea60aaee-d573-45f7-8896-7f22d42df686-audit-log\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.529325 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.529263    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ccw2\" (UniqueName: \"kubernetes.io/projected/ea60aaee-d573-45f7-8896-7f22d42df686-kube-api-access-4ccw2\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.529415 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.529396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-secret-metrics-server-tls\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.529675 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.529651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ea60aaee-d573-45f7-8896-7f22d42df686-metrics-server-audit-profiles\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.529767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.529697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea60aaee-d573-45f7-8896-7f22d42df686-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.529767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.529725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-client-ca-bundle\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.630946 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.630910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ea60aaee-d573-45f7-8896-7f22d42df686-metrics-server-audit-profiles\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.630946 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.630947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea60aaee-d573-45f7-8896-7f22d42df686-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.631165 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.630965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-client-ca-bundle\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.631165 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.630998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-secret-metrics-server-client-certs\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.631165 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.631013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ea60aaee-d573-45f7-8896-7f22d42df686-audit-log\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.631165 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.631035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ccw2\" (UniqueName: \"kubernetes.io/projected/ea60aaee-d573-45f7-8896-7f22d42df686-kube-api-access-4ccw2\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.631165 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.631085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-secret-metrics-server-tls\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.631557 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.631527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ea60aaee-d573-45f7-8896-7f22d42df686-audit-log\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.631804 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.631778    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea60aaee-d573-45f7-8896-7f22d42df686-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.632023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.632004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ea60aaee-d573-45f7-8896-7f22d42df686-metrics-server-audit-profiles\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.633530 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.633509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-secret-metrics-server-client-certs\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.633631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.633615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-secret-metrics-server-tls\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.633686 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.633669    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea60aaee-d573-45f7-8896-7f22d42df686-client-ca-bundle\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.640248 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.640222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ccw2\" (UniqueName: \"kubernetes.io/projected/ea60aaee-d573-45f7-8896-7f22d42df686-kube-api-access-4ccw2\") pod \"metrics-server-58f4f46dfb-dhnq2\" (UID: \"ea60aaee-d573-45f7-8896-7f22d42df686\") " pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.798877 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.798805    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:00.929577 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:00.929553    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"]
Jul 15 05:04:00.932041 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:04:00.932010    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea60aaee_d573_45f7_8896_7f22d42df686.slice/crio-1f5423c2bef07a9f7f99f5df3e81feb15da836dc2b41c0e4599865873e4f1ff6 WatchSource:0}: Error finding container 1f5423c2bef07a9f7f99f5df3e81feb15da836dc2b41c0e4599865873e4f1ff6: Status 404 returned error can't find the container with id 1f5423c2bef07a9f7f99f5df3e81feb15da836dc2b41c0e4599865873e4f1ff6
Jul 15 05:04:01.424098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:01.424059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2" event={"ID":"ea60aaee-d573-45f7-8896-7f22d42df686","Type":"ContainerStarted","Data":"1f5423c2bef07a9f7f99f5df3e81feb15da836dc2b41c0e4599865873e4f1ff6"}
Jul 15 05:04:02.430002 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:02.429965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" event={"ID":"ac839800-6548-4332-98d0-0c5220844e48","Type":"ContainerStarted","Data":"8e300aef671eefe2356d5170da249c0ae3024b4dc1ef8363584e09c90bcc3e52"}
Jul 15 05:04:02.430414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:02.430051    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" event={"ID":"ac839800-6548-4332-98d0-0c5220844e48","Type":"ContainerStarted","Data":"cef1b83341490d206c0da142499e9b0c0d6a33e108543e336e6038e654d9f248"}
Jul 15 05:04:02.430414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:02.430095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" event={"ID":"ac839800-6548-4332-98d0-0c5220844e48","Type":"ContainerStarted","Data":"bbe21382b617d3c4f6702fc68e13acdc430eb04512adb9c26973c15da81105ef"}
Jul 15 05:04:03.434516 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:03.434480    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2" event={"ID":"ea60aaee-d573-45f7-8896-7f22d42df686","Type":"ContainerStarted","Data":"1b224251f63a90506a79c7fbe5935ff49a4e20e4e49c0b85d5bebe2ba621abd6"}
Jul 15 05:04:03.437141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:03.437117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" event={"ID":"ac839800-6548-4332-98d0-0c5220844e48","Type":"ContainerStarted","Data":"62e51d72f0be3e13bb44bd0e48885e09270e9efa39a77e4c83401b905f889306"}
Jul 15 05:04:03.437272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:03.437167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" event={"ID":"ac839800-6548-4332-98d0-0c5220844e48","Type":"ContainerStarted","Data":"3a70d48b9b031fdd25096b676846d4a34844a3e5a2d2d16d0c46249d065fbd6f"}
Jul 15 05:04:03.437272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:03.437191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" event={"ID":"ac839800-6548-4332-98d0-0c5220844e48","Type":"ContainerStarted","Data":"1640695a57599a68db589cd7e2d76a3d939ae01e35d4c2f9b3b10457427378d6"}
Jul 15 05:04:03.437272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:03.437236    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:04:03.453060 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:03.453017    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2" podStartSLOduration=1.703293809 podStartE2EDuration="3.453005503s" totalImagesPullingTime="1.749711694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:04:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:04:03.452065161 +0000 UTC m=+321.619661845" watchObservedRunningTime="2026-07-15 05:04:03.453005503 +0000 UTC m=+321.620602175"
Jul 15 05:04:03.473495 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:03.473457    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps" podStartSLOduration=1.716223878 podStartE2EDuration="5.4734398s" totalImagesPullingTime="3.757215922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:04:03.471731161 +0000 UTC m=+321.639327832" watchObservedRunningTime="2026-07-15 05:04:03.4734398 +0000 UTC m=+321.641036472"
Jul 15 05:04:04.265828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:04.265790    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" podUID="642fc823-a947-4088-baf2-5303557ab537" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 15 05:04:04.265978 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:04.265864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s"
Jul 15 05:04:04.266347 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:04.266316    2581 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"de84a498fb0d9e1cf5ba15289c9f14e28b9283448ab675572f3aa7e687af12ee"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 15 05:04:04.266443 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:04.266364    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" podUID="642fc823-a947-4088-baf2-5303557ab537" containerName="service-proxy" containerID="cri-o://de84a498fb0d9e1cf5ba15289c9f14e28b9283448ab675572f3aa7e687af12ee" gracePeriod=30
Jul 15 05:04:04.442422 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:04.442398    2581 generic.go:356] "Generic (PLEG): container finished" podID="642fc823-a947-4088-baf2-5303557ab537" containerID="de84a498fb0d9e1cf5ba15289c9f14e28b9283448ab675572f3aa7e687af12ee" exitCode=2
Jul 15 05:04:04.442751 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:04.442464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" event={"ID":"642fc823-a947-4088-baf2-5303557ab537","Type":"ContainerDied","Data":"de84a498fb0d9e1cf5ba15289c9f14e28b9283448ab675572f3aa7e687af12ee"}
Jul 15 05:04:04.442751 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:04.442518    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-797f6756f8-5qw6s" event={"ID":"642fc823-a947-4088-baf2-5303557ab537","Type":"ContainerStarted","Data":"3b48a4d5c076ebb013063eef12b0dfea5215b90eb63188f6d20073259e7972a6"}
Jul 15 05:04:09.449147 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:09.449119    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5bdcbcd669-ssmps"
Jul 15 05:04:12.021358 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:12.021323    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77887fdb84-nrd58"]
Jul 15 05:04:20.799730 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:20.799694    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:20.800175 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:20.799760    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:23.742096 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:23.742069    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k9765_6567160b-b7af-4b40-ad93-9e84b7338755/serve-healthcheck-canary/0.log"
Jul 15 05:04:37.041517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.041406    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77887fdb84-nrd58" podUID="b6d8af17-2220-4b6e-ae61-d38236edd083" containerName="console" containerID="cri-o://3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed" gracePeriod=15
Jul 15 05:04:37.282392 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.282370    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:04:37.404779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.404755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-oauth-config\") pod \"b6d8af17-2220-4b6e-ae61-d38236edd083\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") "
Jul 15 05:04:37.404911 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.404802    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zflpj\" (UniqueName: \"kubernetes.io/projected/b6d8af17-2220-4b6e-ae61-d38236edd083-kube-api-access-zflpj\") pod \"b6d8af17-2220-4b6e-ae61-d38236edd083\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") "
Jul 15 05:04:37.404911 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.404837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-service-ca\") pod \"b6d8af17-2220-4b6e-ae61-d38236edd083\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") "
Jul 15 05:04:37.404911 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.404858    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-serving-cert\") pod \"b6d8af17-2220-4b6e-ae61-d38236edd083\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") "
Jul 15 05:04:37.404911 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.404892    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-oauth-serving-cert\") pod \"b6d8af17-2220-4b6e-ae61-d38236edd083\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") "
Jul 15 05:04:37.405100 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.404914    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-trusted-ca-bundle\") pod \"b6d8af17-2220-4b6e-ae61-d38236edd083\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") "
Jul 15 05:04:37.405100 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.404938    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-console-config\") pod \"b6d8af17-2220-4b6e-ae61-d38236edd083\" (UID: \"b6d8af17-2220-4b6e-ae61-d38236edd083\") "
Jul 15 05:04:37.405312 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.405282    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-service-ca" (OuterVolumeSpecName: "service-ca") pod "b6d8af17-2220-4b6e-ae61-d38236edd083" (UID: "b6d8af17-2220-4b6e-ae61-d38236edd083"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:04:37.405442 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.405413    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b6d8af17-2220-4b6e-ae61-d38236edd083" (UID: "b6d8af17-2220-4b6e-ae61-d38236edd083"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:04:37.405509 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.405459    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b6d8af17-2220-4b6e-ae61-d38236edd083" (UID: "b6d8af17-2220-4b6e-ae61-d38236edd083"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:04:37.405509 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.405488    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-console-config" (OuterVolumeSpecName: "console-config") pod "b6d8af17-2220-4b6e-ae61-d38236edd083" (UID: "b6d8af17-2220-4b6e-ae61-d38236edd083"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:04:37.407017 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.406991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b6d8af17-2220-4b6e-ae61-d38236edd083" (UID: "b6d8af17-2220-4b6e-ae61-d38236edd083"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:04:37.407328 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.407309    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b6d8af17-2220-4b6e-ae61-d38236edd083" (UID: "b6d8af17-2220-4b6e-ae61-d38236edd083"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:04:37.407403 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.407334    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d8af17-2220-4b6e-ae61-d38236edd083-kube-api-access-zflpj" (OuterVolumeSpecName: "kube-api-access-zflpj") pod "b6d8af17-2220-4b6e-ae61-d38236edd083" (UID: "b6d8af17-2220-4b6e-ae61-d38236edd083"). InnerVolumeSpecName "kube-api-access-zflpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:04:37.505661 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.505625    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-oauth-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:04:37.505661 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.505656    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zflpj\" (UniqueName: \"kubernetes.io/projected/b6d8af17-2220-4b6e-ae61-d38236edd083-kube-api-access-zflpj\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:04:37.505661 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.505667    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:04:37.505661 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.505675    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6d8af17-2220-4b6e-ae61-d38236edd083-console-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:04:37.505870 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.505684    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-oauth-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:04:37.505870 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.505692    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-trusted-ca-bundle\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:04:37.505870 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.505700    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6d8af17-2220-4b6e-ae61-d38236edd083-console-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:04:37.558681 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.558649    2581 generic.go:356] "Generic (PLEG): container finished" podID="b6d8af17-2220-4b6e-ae61-d38236edd083" containerID="3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed" exitCode=0
Jul 15 05:04:37.558783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.558696    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77887fdb84-nrd58"
Jul 15 05:04:37.558783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.558734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77887fdb84-nrd58" event={"ID":"b6d8af17-2220-4b6e-ae61-d38236edd083","Type":"ContainerDied","Data":"3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed"}
Jul 15 05:04:37.558783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.558780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77887fdb84-nrd58" event={"ID":"b6d8af17-2220-4b6e-ae61-d38236edd083","Type":"ContainerDied","Data":"2cfe982ae55c57564d4033991de813e41694bc93331ef8fe8fce9adb0bac597b"}
Jul 15 05:04:37.558884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.558799    2581 scope.go:117] "RemoveContainer" containerID="3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed"
Jul 15 05:04:37.567656 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.567642    2581 scope.go:117] "RemoveContainer" containerID="3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed"
Jul 15 05:04:37.567904 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:04:37.567885    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed\": container with ID starting with 3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed not found: ID does not exist" containerID="3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed"
Jul 15 05:04:37.567946 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.567912    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed"} err="failed to get container status \"3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed\": rpc error: code = NotFound desc = could not find container \"3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed\": container with ID starting with 3eec89ef477e95a292398b9f03d153171ae97992cfc0e1e79d2f0af93ad39aed not found: ID does not exist"
Jul 15 05:04:37.579304 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.579282    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77887fdb84-nrd58"]
Jul 15 05:04:37.585027 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:37.585008    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77887fdb84-nrd58"]
Jul 15 05:04:38.386380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:38.386348    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6d8af17-2220-4b6e-ae61-d38236edd083" path="/var/lib/kubelet/pods/b6d8af17-2220-4b6e-ae61-d38236edd083/volumes"
Jul 15 05:04:40.804411 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:40.804375    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:40.808039 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:40.808023    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-58f4f46dfb-dhnq2"
Jul 15 05:04:56.331910 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:56.331877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 05:04:56.334078 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:56.334059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971ca253-982f-4e83-86ad-cba13a9970e0-metrics-certs\") pod \"network-metrics-daemon-nx4b2\" (UID: \"971ca253-982f-4e83-86ad-cba13a9970e0\") " pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 05:04:56.385542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:56.385518    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bt4r\""
Jul 15 05:04:56.393658 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:56.393641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nx4b2"
Jul 15 05:04:56.513331 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:56.513306    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nx4b2"]
Jul 15 05:04:56.516217 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:04:56.516168    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971ca253_982f_4e83_86ad_cba13a9970e0.slice/crio-a65702931365a20b72255d37bd535d84e757853001d204fcb19f72a0cfbb887d WatchSource:0}: Error finding container a65702931365a20b72255d37bd535d84e757853001d204fcb19f72a0cfbb887d: Status 404 returned error can't find the container with id a65702931365a20b72255d37bd535d84e757853001d204fcb19f72a0cfbb887d
Jul 15 05:04:56.624074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:56.624015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nx4b2" event={"ID":"971ca253-982f-4e83-86ad-cba13a9970e0","Type":"ContainerStarted","Data":"a65702931365a20b72255d37bd535d84e757853001d204fcb19f72a0cfbb887d"}
Jul 15 05:04:57.628895 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:57.628861    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nx4b2" event={"ID":"971ca253-982f-4e83-86ad-cba13a9970e0","Type":"ContainerStarted","Data":"63cd6b949e331203862e2937f99cbcfd6f0905aa61284d661f9889d4cdd27b95"}
Jul 15 05:04:57.629249 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:57.628939    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nx4b2" event={"ID":"971ca253-982f-4e83-86ad-cba13a9970e0","Type":"ContainerStarted","Data":"b0a9707767dc33397b8806b2710424c89a4b0a89cfa884706ac12c728fb829e0"}
Jul 15 05:04:57.646787 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:04:57.646746    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nx4b2" podStartSLOduration=373.763953157 podStartE2EDuration="6m14.646729845s" totalImagesPullingTime="882.776688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:04:57.645659598 +0000 UTC m=+375.813256270" watchObservedRunningTime="2026-07-15 05:04:57.646729845 +0000 UTC m=+375.814326516"
Jul 15 05:05:24.286360 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.286330    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b74864fb4-t6rz4"]
Jul 15 05:05:24.286715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.286577    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d8af17-2220-4b6e-ae61-d38236edd083" containerName="console"
Jul 15 05:05:24.286715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.286588    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d8af17-2220-4b6e-ae61-d38236edd083" containerName="console"
Jul 15 05:05:24.286715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.286631    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d8af17-2220-4b6e-ae61-d38236edd083" containerName="console"
Jul 15 05:05:24.289547 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.289525    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.293337 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293312    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 05:05:24.293512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293445    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 05:05:24.293608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5pgb9\""
Jul 15 05:05:24.293608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293365    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 05:05:24.293608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293368    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 05:05:24.293773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293380    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 05:05:24.293773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293450    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 05:05:24.293773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.293319    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 05:05:24.302745 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.302717    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b74864fb4-t6rz4"]
Jul 15 05:05:24.303229 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.302765    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 05:05:24.336605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.336566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-trusted-ca-bundle\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.336605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.336601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-serving-cert\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.336828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.336628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddvqg\" (UniqueName: \"kubernetes.io/projected/1235be24-b1a2-429f-8a85-71653092e39d-kube-api-access-ddvqg\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.336828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.336655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-oauth-config\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.336828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.336717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-console-config\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.336828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.336750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-oauth-serving-cert\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.336828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.336777    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-service-ca\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.437594 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.437558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddvqg\" (UniqueName: \"kubernetes.io/projected/1235be24-b1a2-429f-8a85-71653092e39d-kube-api-access-ddvqg\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.437790 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.437611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-oauth-config\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.437790 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.437755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-console-config\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.437902 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.437808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-oauth-serving-cert\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.437902 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.437874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-service-ca\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.437991 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.437941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-trusted-ca-bundle\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.438266 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.438239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-serving-cert\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.438702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.438673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-console-config\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.438702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.438688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-oauth-serving-cert\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.438831 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.438733    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-service-ca\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.439059 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.439042    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-trusted-ca-bundle\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.440111 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.440092    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-oauth-config\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.440476 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.440457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-serving-cert\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.447530 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.447506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddvqg\" (UniqueName: \"kubernetes.io/projected/1235be24-b1a2-429f-8a85-71653092e39d-kube-api-access-ddvqg\") pod \"console-7b74864fb4-t6rz4\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") " pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.604676 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.604644    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:24.747348 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:24.747313    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b74864fb4-t6rz4"]
Jul 15 05:05:24.750301 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:05:24.750272    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1235be24_b1a2_429f_8a85_71653092e39d.slice/crio-4c7ef8efb9d6503e53d10a3d4ab20f6582fd20ccbe4fa59aad9993e17870102f WatchSource:0}: Error finding container 4c7ef8efb9d6503e53d10a3d4ab20f6582fd20ccbe4fa59aad9993e17870102f: Status 404 returned error can't find the container with id 4c7ef8efb9d6503e53d10a3d4ab20f6582fd20ccbe4fa59aad9993e17870102f
Jul 15 05:05:25.724688 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:25.724651    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b74864fb4-t6rz4" event={"ID":"1235be24-b1a2-429f-8a85-71653092e39d","Type":"ContainerStarted","Data":"ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1"}
Jul 15 05:05:25.725152 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:25.724715    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b74864fb4-t6rz4" event={"ID":"1235be24-b1a2-429f-8a85-71653092e39d","Type":"ContainerStarted","Data":"4c7ef8efb9d6503e53d10a3d4ab20f6582fd20ccbe4fa59aad9993e17870102f"}
Jul 15 05:05:25.744406 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:25.744358    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b74864fb4-t6rz4" podStartSLOduration=1.744343403 podStartE2EDuration="1.744343403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:05:25.742995612 +0000 UTC m=+403.910592281" watchObservedRunningTime="2026-07-15 05:05:25.744343403 +0000 UTC m=+403.911940074"
Jul 15 05:05:34.605394 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:34.605359    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:34.605844 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:34.605422    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:34.610267 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:34.610243    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:34.759964 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:34.759934    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:05:34.912806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:34.912726    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65887b9d75-zk6s7"]
Jul 15 05:05:34.915694 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:34.915676    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:34.928800 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:34.928777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65887b9d75-zk6s7"]
Jul 15 05:05:35.013298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.013265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-oauth-serving-cert\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.013298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.013296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfz98\" (UniqueName: \"kubernetes.io/projected/0dac9312-a420-45ef-8654-68ccdad3172d-kube-api-access-sfz98\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.013512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.013333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-oauth-config\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.013512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.013355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-service-ca\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.013512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.013389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-trusted-ca-bundle\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.013512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.013421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-serving-cert\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.013512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.013462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-console-config\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.114668 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.114629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-trusted-ca-bundle\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.114860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.114677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-serving-cert\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.114860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.114719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-console-config\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.114860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.114742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-oauth-serving-cert\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.114860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.114758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfz98\" (UniqueName: \"kubernetes.io/projected/0dac9312-a420-45ef-8654-68ccdad3172d-kube-api-access-sfz98\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.114860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.114791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-oauth-config\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.114860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.114812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-service-ca\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.115589 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.115562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-console-config\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.115659 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.115584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-service-ca\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.115659 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.115642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-trusted-ca-bundle\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.115729 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.115645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-oauth-serving-cert\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.117273 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.117251    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-serving-cert\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.117445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.117430    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-oauth-config\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.124122 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.124083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfz98\" (UniqueName: \"kubernetes.io/projected/0dac9312-a420-45ef-8654-68ccdad3172d-kube-api-access-sfz98\") pod \"console-65887b9d75-zk6s7\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") " pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.225258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.225131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:35.356257 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.356231    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65887b9d75-zk6s7"]
Jul 15 05:05:35.358585 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:05:35.358560    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dac9312_a420_45ef_8654_68ccdad3172d.slice/crio-20dac2ad03ef7c34d98bbefce39dfe85eb5f986b505e9a102ab4c3169938a271 WatchSource:0}: Error finding container 20dac2ad03ef7c34d98bbefce39dfe85eb5f986b505e9a102ab4c3169938a271: Status 404 returned error can't find the container with id 20dac2ad03ef7c34d98bbefce39dfe85eb5f986b505e9a102ab4c3169938a271
Jul 15 05:05:35.761779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.761738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65887b9d75-zk6s7" event={"ID":"0dac9312-a420-45ef-8654-68ccdad3172d","Type":"ContainerStarted","Data":"499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349"}
Jul 15 05:05:35.762190 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.761792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65887b9d75-zk6s7" event={"ID":"0dac9312-a420-45ef-8654-68ccdad3172d","Type":"ContainerStarted","Data":"20dac2ad03ef7c34d98bbefce39dfe85eb5f986b505e9a102ab4c3169938a271"}
Jul 15 05:05:35.781536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:35.781487    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65887b9d75-zk6s7" podStartSLOduration=1.781469274 podStartE2EDuration="1.781469274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:05:35.780474812 +0000 UTC m=+413.948071511" watchObservedRunningTime="2026-07-15 05:05:35.781469274 +0000 UTC m=+413.949065947"
Jul 15 05:05:45.226154 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:45.226114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:45.226636 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:45.226177    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:45.231152 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:45.231128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:45.801414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:45.801382    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:05:45.858084 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:05:45.858042    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b74864fb4-t6rz4"]
Jul 15 05:06:10.879642 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:10.879559    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b74864fb4-t6rz4" podUID="1235be24-b1a2-429f-8a85-71653092e39d" containerName="console" containerID="cri-o://ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1" gracePeriod=15
Jul 15 05:06:11.127536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.127506    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:06:11.203305 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203216    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddvqg\" (UniqueName: \"kubernetes.io/projected/1235be24-b1a2-429f-8a85-71653092e39d-kube-api-access-ddvqg\") pod \"1235be24-b1a2-429f-8a85-71653092e39d\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") "
Jul 15 05:06:11.203305 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203273    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-serving-cert\") pod \"1235be24-b1a2-429f-8a85-71653092e39d\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") "
Jul 15 05:06:11.203305 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-oauth-serving-cert\") pod \"1235be24-b1a2-429f-8a85-71653092e39d\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") "
Jul 15 05:06:11.203566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203330    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-trusted-ca-bundle\") pod \"1235be24-b1a2-429f-8a85-71653092e39d\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") "
Jul 15 05:06:11.203566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203352    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-console-config\") pod \"1235be24-b1a2-429f-8a85-71653092e39d\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") "
Jul 15 05:06:11.203566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203371    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-service-ca\") pod \"1235be24-b1a2-429f-8a85-71653092e39d\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") "
Jul 15 05:06:11.203566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203397    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-oauth-config\") pod \"1235be24-b1a2-429f-8a85-71653092e39d\" (UID: \"1235be24-b1a2-429f-8a85-71653092e39d\") "
Jul 15 05:06:11.203848 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203815    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1235be24-b1a2-429f-8a85-71653092e39d" (UID: "1235be24-b1a2-429f-8a85-71653092e39d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:11.204014 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203853    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-console-config" (OuterVolumeSpecName: "console-config") pod "1235be24-b1a2-429f-8a85-71653092e39d" (UID: "1235be24-b1a2-429f-8a85-71653092e39d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:11.204014 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203902    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-service-ca" (OuterVolumeSpecName: "service-ca") pod "1235be24-b1a2-429f-8a85-71653092e39d" (UID: "1235be24-b1a2-429f-8a85-71653092e39d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:11.204014 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.203930    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1235be24-b1a2-429f-8a85-71653092e39d" (UID: "1235be24-b1a2-429f-8a85-71653092e39d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:11.205480 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.205457    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1235be24-b1a2-429f-8a85-71653092e39d-kube-api-access-ddvqg" (OuterVolumeSpecName: "kube-api-access-ddvqg") pod "1235be24-b1a2-429f-8a85-71653092e39d" (UID: "1235be24-b1a2-429f-8a85-71653092e39d"). InnerVolumeSpecName "kube-api-access-ddvqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:06:11.205480 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.205471    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1235be24-b1a2-429f-8a85-71653092e39d" (UID: "1235be24-b1a2-429f-8a85-71653092e39d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:06:11.205601 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.205541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1235be24-b1a2-429f-8a85-71653092e39d" (UID: "1235be24-b1a2-429f-8a85-71653092e39d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:06:11.304879 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.304841    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-oauth-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:06:11.304879 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.304869    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-trusted-ca-bundle\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:06:11.304879 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.304880    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-console-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:06:11.304879 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.304889    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1235be24-b1a2-429f-8a85-71653092e39d-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:06:11.305171 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.304896    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-oauth-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:06:11.305171 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.304904    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddvqg\" (UniqueName: \"kubernetes.io/projected/1235be24-b1a2-429f-8a85-71653092e39d-kube-api-access-ddvqg\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:06:11.305171 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.304913    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1235be24-b1a2-429f-8a85-71653092e39d-console-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:06:11.889731 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.889691    2581 generic.go:356] "Generic (PLEG): container finished" podID="1235be24-b1a2-429f-8a85-71653092e39d" containerID="ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1" exitCode=0
Jul 15 05:06:11.890164 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.889739    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b74864fb4-t6rz4"
Jul 15 05:06:11.890164 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.889762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b74864fb4-t6rz4" event={"ID":"1235be24-b1a2-429f-8a85-71653092e39d","Type":"ContainerDied","Data":"ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1"}
Jul 15 05:06:11.890164 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.889804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b74864fb4-t6rz4" event={"ID":"1235be24-b1a2-429f-8a85-71653092e39d","Type":"ContainerDied","Data":"4c7ef8efb9d6503e53d10a3d4ab20f6582fd20ccbe4fa59aad9993e17870102f"}
Jul 15 05:06:11.890164 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.889824    2581 scope.go:117] "RemoveContainer" containerID="ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1"
Jul 15 05:06:11.898246 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.898222    2581 scope.go:117] "RemoveContainer" containerID="ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1"
Jul 15 05:06:11.898524 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:06:11.898507    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1\": container with ID starting with ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1 not found: ID does not exist" containerID="ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1"
Jul 15 05:06:11.898578 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.898533    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1"} err="failed to get container status \"ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1\": rpc error: code = NotFound desc = could not find container \"ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1\": container with ID starting with ae012acf9d2be131439e09dd7cf18deed379c03c0c22f3819027205a66f47ec1 not found: ID does not exist"
Jul 15 05:06:11.911511 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.911483    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b74864fb4-t6rz4"]
Jul 15 05:06:11.913889 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:11.913862    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b74864fb4-t6rz4"]
Jul 15 05:06:12.387320 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:06:12.387279    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1235be24-b1a2-429f-8a85-71653092e39d" path="/var/lib/kubelet/pods/1235be24-b1a2-429f-8a85-71653092e39d/volumes"
Jul 15 05:07:17.318284 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.318255    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"]
Jul 15 05:07:17.318677 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.318514    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1235be24-b1a2-429f-8a85-71653092e39d" containerName="console"
Jul 15 05:07:17.318677 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.318524    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1235be24-b1a2-429f-8a85-71653092e39d" containerName="console"
Jul 15 05:07:17.318677 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.318565    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1235be24-b1a2-429f-8a85-71653092e39d" containerName="console"
Jul 15 05:07:17.321288 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.321264    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.324663 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.324642    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f6kzc\""
Jul 15 05:07:17.324774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.324663    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 15 05:07:17.324951 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.324936    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 15 05:07:17.369004 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.368983    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"]
Jul 15 05:07:17.481042 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.481019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e242b363-8c73-4dd5-aeb4-fbc32b293b07-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-988kb\" (UID: \"e242b363-8c73-4dd5-aeb4-fbc32b293b07\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.481167 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.481053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz9vv\" (UniqueName: \"kubernetes.io/projected/e242b363-8c73-4dd5-aeb4-fbc32b293b07-kube-api-access-cz9vv\") pod \"cert-manager-operator-controller-manager-55c444bf86-988kb\" (UID: \"e242b363-8c73-4dd5-aeb4-fbc32b293b07\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.581428 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.581370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz9vv\" (UniqueName: \"kubernetes.io/projected/e242b363-8c73-4dd5-aeb4-fbc32b293b07-kube-api-access-cz9vv\") pod \"cert-manager-operator-controller-manager-55c444bf86-988kb\" (UID: \"e242b363-8c73-4dd5-aeb4-fbc32b293b07\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.581507 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.581429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e242b363-8c73-4dd5-aeb4-fbc32b293b07-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-988kb\" (UID: \"e242b363-8c73-4dd5-aeb4-fbc32b293b07\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.581769 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.581751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e242b363-8c73-4dd5-aeb4-fbc32b293b07-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-988kb\" (UID: \"e242b363-8c73-4dd5-aeb4-fbc32b293b07\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.594988 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.594958    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz9vv\" (UniqueName: \"kubernetes.io/projected/e242b363-8c73-4dd5-aeb4-fbc32b293b07-kube-api-access-cz9vv\") pod \"cert-manager-operator-controller-manager-55c444bf86-988kb\" (UID: \"e242b363-8c73-4dd5-aeb4-fbc32b293b07\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.630935 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.630913    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"
Jul 15 05:07:17.758535 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:17.758507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb"]
Jul 15 05:07:17.762161 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:07:17.762129    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode242b363_8c73_4dd5_aeb4_fbc32b293b07.slice/crio-9428260cdb07203637f659bb06a565ee94deaf42bf2de73412e960ed7996b50e WatchSource:0}: Error finding container 9428260cdb07203637f659bb06a565ee94deaf42bf2de73412e960ed7996b50e: Status 404 returned error can't find the container with id 9428260cdb07203637f659bb06a565ee94deaf42bf2de73412e960ed7996b50e
Jul 15 05:07:18.118181 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:18.118150    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb" event={"ID":"e242b363-8c73-4dd5-aeb4-fbc32b293b07","Type":"ContainerStarted","Data":"9428260cdb07203637f659bb06a565ee94deaf42bf2de73412e960ed7996b50e"}
Jul 15 05:07:22.135726 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:22.135693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb" event={"ID":"e242b363-8c73-4dd5-aeb4-fbc32b293b07","Type":"ContainerStarted","Data":"4a7f5d67b68cfbbca3294001a0f03de29ce170554162902b8df555461aba4eec"}
Jul 15 05:07:22.160425 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:22.160373    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-988kb" podStartSLOduration=1.7041023530000001 podStartE2EDuration="5.16035918s" totalImagesPullingTime="3.456256827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:07:22.157746812 +0000 UTC m=+520.325343483" watchObservedRunningTime="2026-07-15 05:07:22.16035918 +0000 UTC m=+520.327955852"
Jul 15 05:07:33.520815 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.520782    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8j6fl"]
Jul 15 05:07:33.524040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.524021    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.526989 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.526969    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 05:07:33.528027 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.528010    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 05:07:33.528092 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.528015    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5z4sg\""
Jul 15 05:07:33.534129 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.534108    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8j6fl"]
Jul 15 05:07:33.696281 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.696242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c0b693-e6e5-494f-a665-298b83fce00c-bound-sa-token\") pod \"cert-manager-55b4d954d5-8j6fl\" (UID: \"24c0b693-e6e5-494f-a665-298b83fce00c\") " pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.696281 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.696286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw9jj\" (UniqueName: \"kubernetes.io/projected/24c0b693-e6e5-494f-a665-298b83fce00c-kube-api-access-lw9jj\") pod \"cert-manager-55b4d954d5-8j6fl\" (UID: \"24c0b693-e6e5-494f-a665-298b83fce00c\") " pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.796947 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.796872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c0b693-e6e5-494f-a665-298b83fce00c-bound-sa-token\") pod \"cert-manager-55b4d954d5-8j6fl\" (UID: \"24c0b693-e6e5-494f-a665-298b83fce00c\") " pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.796947 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.796902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw9jj\" (UniqueName: \"kubernetes.io/projected/24c0b693-e6e5-494f-a665-298b83fce00c-kube-api-access-lw9jj\") pod \"cert-manager-55b4d954d5-8j6fl\" (UID: \"24c0b693-e6e5-494f-a665-298b83fce00c\") " pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.817115 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.817090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24c0b693-e6e5-494f-a665-298b83fce00c-bound-sa-token\") pod \"cert-manager-55b4d954d5-8j6fl\" (UID: \"24c0b693-e6e5-494f-a665-298b83fce00c\") " pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.817262 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.817185    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw9jj\" (UniqueName: \"kubernetes.io/projected/24c0b693-e6e5-494f-a665-298b83fce00c-kube-api-access-lw9jj\") pod \"cert-manager-55b4d954d5-8j6fl\" (UID: \"24c0b693-e6e5-494f-a665-298b83fce00c\") " pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.831913 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.831892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8j6fl"
Jul 15 05:07:33.955658 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:33.955637    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8j6fl"]
Jul 15 05:07:33.957747 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:07:33.957723    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24c0b693_e6e5_494f_a665_298b83fce00c.slice/crio-c19878d84460deea8bd51bf15b496e31353af250547b85c266fcb38ba2ddd77d WatchSource:0}: Error finding container c19878d84460deea8bd51bf15b496e31353af250547b85c266fcb38ba2ddd77d: Status 404 returned error can't find the container with id c19878d84460deea8bd51bf15b496e31353af250547b85c266fcb38ba2ddd77d
Jul 15 05:07:34.185643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:34.185608    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8j6fl" event={"ID":"24c0b693-e6e5-494f-a665-298b83fce00c","Type":"ContainerStarted","Data":"c19878d84460deea8bd51bf15b496e31353af250547b85c266fcb38ba2ddd77d"}
Jul 15 05:07:37.198555 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:37.198519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8j6fl" event={"ID":"24c0b693-e6e5-494f-a665-298b83fce00c","Type":"ContainerStarted","Data":"60023f763654f57912bd96ece7f7e6fae27979c39990bddd2d5b33a999541559"}
Jul 15 05:07:37.214506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:07:37.214464    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8j6fl" podStartSLOduration=1.848428217 podStartE2EDuration="4.214447928s" totalImagesPullingTime="2.366019711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:07:37.21361692 +0000 UTC m=+535.381213592" watchObservedRunningTime="2026-07-15 05:07:37.214447928 +0000 UTC m=+535.382044599"
Jul 15 05:08:03.468081 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.468047    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"]
Jul 15 05:08:03.473611 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.473592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.478487 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.478469    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 05:08:03.478574 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.478505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k6dtd\""
Jul 15 05:08:03.479016 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.478995    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 05:08:03.479016 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.479012    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 05:08:03.482070 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.482051    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 05:08:03.491141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.491123    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"]
Jul 15 05:08:03.498684 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.498665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/607f6c72-cd50-402e-93d2-fab7ebb58980-tmp\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.498780 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.498715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/607f6c72-cd50-402e-93d2-fab7ebb58980-manifests\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.498780 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.498738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/607f6c72-cd50-402e-93d2-fab7ebb58980-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.498780 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.498764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlfm2\" (UniqueName: \"kubernetes.io/projected/607f6c72-cd50-402e-93d2-fab7ebb58980-kube-api-access-xlfm2\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.498895 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.498805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/607f6c72-cd50-402e-93d2-fab7ebb58980-webhook-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.599078 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.599051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/607f6c72-cd50-402e-93d2-fab7ebb58980-manifests\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.599192 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.599090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/607f6c72-cd50-402e-93d2-fab7ebb58980-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.599192 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.599117    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlfm2\" (UniqueName: \"kubernetes.io/projected/607f6c72-cd50-402e-93d2-fab7ebb58980-kube-api-access-xlfm2\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.599192 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.599147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/607f6c72-cd50-402e-93d2-fab7ebb58980-webhook-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.599364 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.599193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/607f6c72-cd50-402e-93d2-fab7ebb58980-tmp\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.599734 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.599699    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/607f6c72-cd50-402e-93d2-fab7ebb58980-manifests\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.601532 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.601516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/607f6c72-cd50-402e-93d2-fab7ebb58980-tmp\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.602615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.602600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/607f6c72-cd50-402e-93d2-fab7ebb58980-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.602778 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.602758    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/607f6c72-cd50-402e-93d2-fab7ebb58980-webhook-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.612491 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.612469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlfm2\" (UniqueName: \"kubernetes.io/projected/607f6c72-cd50-402e-93d2-fab7ebb58980-kube-api-access-xlfm2\") pod \"opendatahub-operator-controller-manager-749c75689b-mqwp6\" (UID: \"607f6c72-cd50-402e-93d2-fab7ebb58980\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.783922 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.783872    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:03.932360 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:03.932329    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"]
Jul 15 05:08:03.934907 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:08:03.934881    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod607f6c72_cd50_402e_93d2_fab7ebb58980.slice/crio-14bd4f03b26b36346077710276b8e73be5beb7d45279fcecf1bd7938583f88d6 WatchSource:0}: Error finding container 14bd4f03b26b36346077710276b8e73be5beb7d45279fcecf1bd7938583f88d6: Status 404 returned error can't find the container with id 14bd4f03b26b36346077710276b8e73be5beb7d45279fcecf1bd7938583f88d6
Jul 15 05:08:04.309635 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:04.309603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6" event={"ID":"607f6c72-cd50-402e-93d2-fab7ebb58980","Type":"ContainerStarted","Data":"14bd4f03b26b36346077710276b8e73be5beb7d45279fcecf1bd7938583f88d6"}
Jul 15 05:08:07.324464 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:07.324423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6" event={"ID":"607f6c72-cd50-402e-93d2-fab7ebb58980","Type":"ContainerStarted","Data":"3118f5fca4a64c805958371170957b907d529627bcddc779479d77be92392422"}
Jul 15 05:08:08.330312 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:08.330273    2581 generic.go:356] "Generic (PLEG): container finished" podID="607f6c72-cd50-402e-93d2-fab7ebb58980" containerID="3118f5fca4a64c805958371170957b907d529627bcddc779479d77be92392422" exitCode=0
Jul 15 05:08:08.330312 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:08.330312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6" event={"ID":"607f6c72-cd50-402e-93d2-fab7ebb58980","Type":"ContainerDied","Data":"3118f5fca4a64c805958371170957b907d529627bcddc779479d77be92392422"}
Jul 15 05:08:09.336695 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:09.336657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6" event={"ID":"607f6c72-cd50-402e-93d2-fab7ebb58980","Type":"ContainerStarted","Data":"7a19f0b20b121934c42e31a0e2b637e8624b8af0b3433309e15f82e2c350b747"}
Jul 15 05:08:09.337069 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:09.336755    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:09.359625 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:09.359581    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6" podStartSLOduration=2.070432662 podStartE2EDuration="6.359566621s" totalImagesPullingTime="3.264754092s" totalInitContainerRuntime="1.024379867s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:09.358184849 +0000 UTC m=+567.525781512" watchObservedRunningTime="2026-07-15 05:08:09.359566621 +0000 UTC m=+567.527163292"
Jul 15 05:08:20.343849 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:20.343814    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-mqwp6"
Jul 15 05:08:24.318662 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.318623    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-222x5"]
Jul 15 05:08:24.327152 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.327131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:24.329798 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.329775    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zj7gc\""
Jul 15 05:08:24.330068 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.329793    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 15 05:08:24.330709 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.330680    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-222x5"]
Jul 15 05:08:24.461779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.461749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fed6480-2f80-42b3-88c1-7fd28c1a4341-cert\") pod \"odh-model-controller-7998c67dc4-222x5\" (UID: \"5fed6480-2f80-42b3-88c1-7fd28c1a4341\") " pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:24.461931 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.461872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf24t\" (UniqueName: \"kubernetes.io/projected/5fed6480-2f80-42b3-88c1-7fd28c1a4341-kube-api-access-qf24t\") pod \"odh-model-controller-7998c67dc4-222x5\" (UID: \"5fed6480-2f80-42b3-88c1-7fd28c1a4341\") " pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:24.562496 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.562466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fed6480-2f80-42b3-88c1-7fd28c1a4341-cert\") pod \"odh-model-controller-7998c67dc4-222x5\" (UID: \"5fed6480-2f80-42b3-88c1-7fd28c1a4341\") " pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:24.562648 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.562524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf24t\" (UniqueName: \"kubernetes.io/projected/5fed6480-2f80-42b3-88c1-7fd28c1a4341-kube-api-access-qf24t\") pod \"odh-model-controller-7998c67dc4-222x5\" (UID: \"5fed6480-2f80-42b3-88c1-7fd28c1a4341\") " pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:24.562648 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:08:24.562609    2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 05:08:24.563128 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:08:24.562682    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fed6480-2f80-42b3-88c1-7fd28c1a4341-cert podName:5fed6480-2f80-42b3-88c1-7fd28c1a4341 nodeName:}" failed. No retries permitted until 2026-07-15 05:08:25.06266585 +0000 UTC m=+583.230262500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5fed6480-2f80-42b3-88c1-7fd28c1a4341-cert") pod "odh-model-controller-7998c67dc4-222x5" (UID: "5fed6480-2f80-42b3-88c1-7fd28c1a4341") : secret "odh-model-controller-webhook-cert" not found
Jul 15 05:08:24.578758 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.578691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf24t\" (UniqueName: \"kubernetes.io/projected/5fed6480-2f80-42b3-88c1-7fd28c1a4341-kube-api-access-qf24t\") pod \"odh-model-controller-7998c67dc4-222x5\" (UID: \"5fed6480-2f80-42b3-88c1-7fd28c1a4341\") " pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:24.858620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.858546    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"]
Jul 15 05:08:24.861827 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.861811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:24.864464 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.864437    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 05:08:24.864638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.864623    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 05:08:24.864888 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.864863    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 05:08:24.864986 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.864902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 05:08:24.864986 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.864915    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cc22q\""
Jul 15 05:08:24.870125 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.870107    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"]
Jul 15 05:08:24.965730 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.965697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/949df993-3e3a-4c6a-9980-a72574ee759d-tls-certs\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:24.965875 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.965762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/949df993-3e3a-4c6a-9980-a72574ee759d-tmp\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:24.965875 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:24.965844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6qrf\" (UniqueName: \"kubernetes.io/projected/949df993-3e3a-4c6a-9980-a72574ee759d-kube-api-access-k6qrf\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.066480 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.066451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6qrf\" (UniqueName: \"kubernetes.io/projected/949df993-3e3a-4c6a-9980-a72574ee759d-kube-api-access-k6qrf\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.066632 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.066487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/949df993-3e3a-4c6a-9980-a72574ee759d-tls-certs\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.066632 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.066509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fed6480-2f80-42b3-88c1-7fd28c1a4341-cert\") pod \"odh-model-controller-7998c67dc4-222x5\" (UID: \"5fed6480-2f80-42b3-88c1-7fd28c1a4341\") " pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:25.066632 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.066557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/949df993-3e3a-4c6a-9980-a72574ee759d-tmp\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.068784 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.068754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/949df993-3e3a-4c6a-9980-a72574ee759d-tmp\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.068902 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.068885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/949df993-3e3a-4c6a-9980-a72574ee759d-tls-certs\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.068944 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.068905    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fed6480-2f80-42b3-88c1-7fd28c1a4341-cert\") pod \"odh-model-controller-7998c67dc4-222x5\" (UID: \"5fed6480-2f80-42b3-88c1-7fd28c1a4341\") " pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:25.074071 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.074048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6qrf\" (UniqueName: \"kubernetes.io/projected/949df993-3e3a-4c6a-9980-a72574ee759d-kube-api-access-k6qrf\") pod \"kube-auth-proxy-57bf8bf567-wndzn\" (UID: \"949df993-3e3a-4c6a-9980-a72574ee759d\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.105949 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.105923    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dvhc6"]
Jul 15 05:08:25.109182 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.109135    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.111983 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.111968    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 15 05:08:25.121186 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.121165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dvhc6"]
Jul 15 05:08:25.167818 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.167796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6fbm\" (UniqueName: \"kubernetes.io/projected/2aa0e25e-b678-48cb-b7fb-dec84b6844ce-kube-api-access-w6fbm\") pod \"model-serving-api-54d89bd5fc-dvhc6\" (UID: \"2aa0e25e-b678-48cb-b7fb-dec84b6844ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.167933 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.167835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aa0e25e-b678-48cb-b7fb-dec84b6844ce-tls-certs\") pod \"model-serving-api-54d89bd5fc-dvhc6\" (UID: \"2aa0e25e-b678-48cb-b7fb-dec84b6844ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.171731 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.171715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"
Jul 15 05:08:25.238939 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.238912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:25.268762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.268733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6fbm\" (UniqueName: \"kubernetes.io/projected/2aa0e25e-b678-48cb-b7fb-dec84b6844ce-kube-api-access-w6fbm\") pod \"model-serving-api-54d89bd5fc-dvhc6\" (UID: \"2aa0e25e-b678-48cb-b7fb-dec84b6844ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.268883 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.268786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aa0e25e-b678-48cb-b7fb-dec84b6844ce-tls-certs\") pod \"model-serving-api-54d89bd5fc-dvhc6\" (UID: \"2aa0e25e-b678-48cb-b7fb-dec84b6844ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.271303 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.271280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aa0e25e-b678-48cb-b7fb-dec84b6844ce-tls-certs\") pod \"model-serving-api-54d89bd5fc-dvhc6\" (UID: \"2aa0e25e-b678-48cb-b7fb-dec84b6844ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.278559 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.278528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6fbm\" (UniqueName: \"kubernetes.io/projected/2aa0e25e-b678-48cb-b7fb-dec84b6844ce-kube-api-access-w6fbm\") pod \"model-serving-api-54d89bd5fc-dvhc6\" (UID: \"2aa0e25e-b678-48cb-b7fb-dec84b6844ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.309624 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.309538    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn"]
Jul 15 05:08:25.313191 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:08:25.313152    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949df993_3e3a_4c6a_9980_a72574ee759d.slice/crio-d686fb7392a6c6ce6b14c21f53f3bb0da938b556f995a6eb9b5e5bafe642ed4a WatchSource:0}: Error finding container d686fb7392a6c6ce6b14c21f53f3bb0da938b556f995a6eb9b5e5bafe642ed4a: Status 404 returned error can't find the container with id d686fb7392a6c6ce6b14c21f53f3bb0da938b556f995a6eb9b5e5bafe642ed4a
Jul 15 05:08:25.381889 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.381777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-222x5"]
Jul 15 05:08:25.384672 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:08:25.384646    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fed6480_2f80_42b3_88c1_7fd28c1a4341.slice/crio-4090540c46b8471f1a5485172a95669b253ec3814e6c69f3ed0de7eba1420ab5 WatchSource:0}: Error finding container 4090540c46b8471f1a5485172a95669b253ec3814e6c69f3ed0de7eba1420ab5: Status 404 returned error can't find the container with id 4090540c46b8471f1a5485172a95669b253ec3814e6c69f3ed0de7eba1420ab5
Jul 15 05:08:25.403100 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.403073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-222x5" event={"ID":"5fed6480-2f80-42b3-88c1-7fd28c1a4341","Type":"ContainerStarted","Data":"4090540c46b8471f1a5485172a95669b253ec3814e6c69f3ed0de7eba1420ab5"}
Jul 15 05:08:25.404311 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.404285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn" event={"ID":"949df993-3e3a-4c6a-9980-a72574ee759d","Type":"ContainerStarted","Data":"d686fb7392a6c6ce6b14c21f53f3bb0da938b556f995a6eb9b5e5bafe642ed4a"}
Jul 15 05:08:25.421550 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.421520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:25.563251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.563230    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dvhc6"]
Jul 15 05:08:25.565419 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:08:25.565397    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aa0e25e_b678_48cb_b7fb_dec84b6844ce.slice/crio-bba1345a98743144a3a5dc8690195a1d5e742f363253eda89cc99a60d038c389 WatchSource:0}: Error finding container bba1345a98743144a3a5dc8690195a1d5e742f363253eda89cc99a60d038c389: Status 404 returned error can't find the container with id bba1345a98743144a3a5dc8690195a1d5e742f363253eda89cc99a60d038c389
Jul 15 05:08:25.920650 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.920616    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk268"]
Jul 15 05:08:25.925251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.925231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:25.929760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.929736    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 05:08:25.929760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.929751    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 05:08:25.929760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.929739    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 05:08:25.930002 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.929822    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 05:08:25.930099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.930083    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m9cmb\""
Jul 15 05:08:25.941455 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.941435    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk268"]
Jul 15 05:08:25.975720 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.975694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bvbn\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-kube-api-access-8bvbn\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:25.975823 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.975741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-istio-token\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:25.975823 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.975783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:25.975940 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.975851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b950ef-9820-4101-adba-7898cc6ab74c-local-certs\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:25.975940 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.975899    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-cacerts\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:25.975940 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.975921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:25.976098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:25.975953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.076743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.076686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-istio-token\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.076743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.076735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.077001 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.076802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b950ef-9820-4101-adba-7898cc6ab74c-local-certs\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.077111 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.077004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-cacerts\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.077111 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.077041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.077111 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.077073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.077297 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.077116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bvbn\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-kube-api-access-8bvbn\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.078392 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.078367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.080271 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.080225    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-cacerts\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.080460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.080439    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b950ef-9820-4101-adba-7898cc6ab74c-local-certs\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.081156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.081120    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.081467 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.081447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.090757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.090712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bvbn\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-kube-api-access-8bvbn\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.090898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.090797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-istio-token\") pod \"istiod-openshift-gateway-7574459656-lk268\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.237217 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.237041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:26.413633 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.413510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6" event={"ID":"2aa0e25e-b678-48cb-b7fb-dec84b6844ce","Type":"ContainerStarted","Data":"bba1345a98743144a3a5dc8690195a1d5e742f363253eda89cc99a60d038c389"}
Jul 15 05:08:26.424429 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:26.424076    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk268"]
Jul 15 05:08:26.426287 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:08:26.426253    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b950ef_9820_4101_adba_7898cc6ab74c.slice/crio-df31669c45b51c2c5cad19b4793cc2b2e938b73bb06047086763d5cafad565aa WatchSource:0}: Error finding container df31669c45b51c2c5cad19b4793cc2b2e938b73bb06047086763d5cafad565aa: Status 404 returned error can't find the container with id df31669c45b51c2c5cad19b4793cc2b2e938b73bb06047086763d5cafad565aa
Jul 15 05:08:27.423441 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:27.423402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268" event={"ID":"75b950ef-9820-4101-adba-7898cc6ab74c","Type":"ContainerStarted","Data":"df31669c45b51c2c5cad19b4793cc2b2e938b73bb06047086763d5cafad565aa"}
Jul 15 05:08:30.439973 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.439933    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn" event={"ID":"949df993-3e3a-4c6a-9980-a72574ee759d","Type":"ContainerStarted","Data":"5d4ca5355b8f6393d2f99851434118bb7f2df79710c67bed2fe52b0ecc6600e8"}
Jul 15 05:08:30.441450 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.441421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-222x5" event={"ID":"5fed6480-2f80-42b3-88c1-7fd28c1a4341","Type":"ContainerStarted","Data":"ddf11c2b2b56d246c02fa09fa0b25ef5ff75541a07a1912b79c0ab037ddf5a4d"}
Jul 15 05:08:30.441584 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.441524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:30.442843 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.442822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6" event={"ID":"2aa0e25e-b678-48cb-b7fb-dec84b6844ce","Type":"ContainerStarted","Data":"cfd7eca46e8bbcd45bb9f260cb16956bf07f58d5cafdbefcb55bbb89eed3942d"}
Jul 15 05:08:30.443005 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.442992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:30.469847 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.469798    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-wndzn" podStartSLOduration=2.090542077 podStartE2EDuration="6.469785322s" totalImagesPullingTime="4.379243245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:30.468950418 +0000 UTC m=+588.636547090" watchObservedRunningTime="2026-07-15 05:08:30.469785322 +0000 UTC m=+588.637381994"
Jul 15 05:08:30.497287 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.497243    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-222x5" podStartSLOduration=2.197010369 podStartE2EDuration="6.497232096s" totalImagesPullingTime="4.300221727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:30.496494934 +0000 UTC m=+588.664091607" watchObservedRunningTime="2026-07-15 05:08:30.497232096 +0000 UTC m=+588.664828763"
Jul 15 05:08:30.523006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:30.522964    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6" podStartSLOduration=1.4014306859999999 podStartE2EDuration="5.52295217s" totalImagesPullingTime="4.121521484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:30.521603068 +0000 UTC m=+588.689199742" watchObservedRunningTime="2026-07-15 05:08:30.52295217 +0000 UTC m=+588.690548879"
Jul 15 05:08:33.726745 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:33.726707    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:08:33.726984 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:33.726780    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:08:34.461308 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.461274    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268" event={"ID":"75b950ef-9820-4101-adba-7898cc6ab74c","Type":"ContainerStarted","Data":"69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b"}
Jul 15 05:08:34.461536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.461330    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:34.462891 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.462863    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-lk268 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 05:08:34.462992 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.462913    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268" podUID="75b950ef-9820-4101-adba-7898cc6ab74c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 05:08:34.528680 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.528617    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268" podStartSLOduration=2.230833525 podStartE2EDuration="9.528596517s" totalImagesPullingTime="7.297762992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:34.526873167 +0000 UTC m=+592.694469833" watchObservedRunningTime="2026-07-15 05:08:34.528596517 +0000 UTC m=+592.696193189"
Jul 15 05:08:34.836705 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.836676    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"]
Jul 15 05:08:34.839911 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.839891    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.845149 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.845128    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-n6d9x\""
Jul 15 05:08:34.876053 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.876027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"]
Jul 15 05:08:34.959082 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959252 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959252 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959252 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959417 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959417 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959417 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959370    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959417 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959399    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hszgl\" (UniqueName: \"kubernetes.io/projected/d0b7352d-a5ae-4e48-aa10-919032c86c1f-kube-api-access-hszgl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:34.959555 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:34.959497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060410 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060894 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hszgl\" (UniqueName: \"kubernetes.io/projected/d0b7352d-a5ae-4e48-aa10-919032c86c1f-kube-api-access-hszgl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060894 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.060894 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.061039 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.060964    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.061197 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.061178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.061304 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.061254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.062808 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.062791    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.063023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.063007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.086988 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.086921    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0b7352d-a5ae-4e48-aa10-919032c86c1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.087092 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.087077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hszgl\" (UniqueName: \"kubernetes.io/projected/d0b7352d-a5ae-4e48-aa10-919032c86c1f-kube-api-access-hszgl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s7h8g\" (UID: \"d0b7352d-a5ae-4e48-aa10-919032c86c1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.150052 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.150023    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:35.293609 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.293580    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"]
Jul 15 05:08:35.296103 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:08:35.296075    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b7352d_a5ae_4e48_aa10_919032c86c1f.slice/crio-40467fbc8b58cb722fec0ab70415d3bb018c127503f3ec2e978a6140d3101f75 WatchSource:0}: Error finding container 40467fbc8b58cb722fec0ab70415d3bb018c127503f3ec2e978a6140d3101f75: Status 404 returned error can't find the container with id 40467fbc8b58cb722fec0ab70415d3bb018c127503f3ec2e978a6140d3101f75
Jul 15 05:08:35.466117 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.466038    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g" event={"ID":"d0b7352d-a5ae-4e48-aa10-919032c86c1f","Type":"ContainerStarted","Data":"40467fbc8b58cb722fec0ab70415d3bb018c127503f3ec2e978a6140d3101f75"}
Jul 15 05:08:35.466742 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.466720    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:35.500371 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.500344    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"]
Jul 15 05:08:35.504364 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.504345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:35.507831 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.507799    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hb96x\""
Jul 15 05:08:35.508822 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.508805    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 15 05:08:35.509892 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.509873    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"]
Jul 15 05:08:35.512893 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.512878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:35.515401 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.515381    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 15 05:08:35.515499 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.515406    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jtbtw\""
Jul 15 05:08:35.526176 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.526155    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"]
Jul 15 05:08:35.548737 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.548714    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"]
Jul 15 05:08:35.665923 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.665897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtsg\" (UniqueName: \"kubernetes.io/projected/3b47826b-4ca2-431a-a537-e77aa23881f1-kube-api-access-4xtsg\") pod \"kserve-controller-manager-d6f4b65b8-92dw4\" (UID: \"3b47826b-4ca2-431a-a537-e77aa23881f1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:35.665923 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.665928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9qcg\" (UniqueName: \"kubernetes.io/projected/25b6a7f1-e50a-4e38-a0db-06739cd7715e-kube-api-access-j9qcg\") pod \"llmisvc-controller-manager-777c6bd7d5-g4g9w\" (UID: \"25b6a7f1-e50a-4e38-a0db-06739cd7715e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:35.666098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.665962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b47826b-4ca2-431a-a537-e77aa23881f1-cert\") pod \"kserve-controller-manager-d6f4b65b8-92dw4\" (UID: \"3b47826b-4ca2-431a-a537-e77aa23881f1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:35.666098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.665979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25b6a7f1-e50a-4e38-a0db-06739cd7715e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g4g9w\" (UID: \"25b6a7f1-e50a-4e38-a0db-06739cd7715e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:35.767030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.766956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtsg\" (UniqueName: \"kubernetes.io/projected/3b47826b-4ca2-431a-a537-e77aa23881f1-kube-api-access-4xtsg\") pod \"kserve-controller-manager-d6f4b65b8-92dw4\" (UID: \"3b47826b-4ca2-431a-a537-e77aa23881f1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:35.767030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.766991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9qcg\" (UniqueName: \"kubernetes.io/projected/25b6a7f1-e50a-4e38-a0db-06739cd7715e-kube-api-access-j9qcg\") pod \"llmisvc-controller-manager-777c6bd7d5-g4g9w\" (UID: \"25b6a7f1-e50a-4e38-a0db-06739cd7715e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:35.767030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.767026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b47826b-4ca2-431a-a537-e77aa23881f1-cert\") pod \"kserve-controller-manager-d6f4b65b8-92dw4\" (UID: \"3b47826b-4ca2-431a-a537-e77aa23881f1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:35.767251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.767041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25b6a7f1-e50a-4e38-a0db-06739cd7715e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g4g9w\" (UID: \"25b6a7f1-e50a-4e38-a0db-06739cd7715e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:35.767251 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:08:35.767145    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 15 05:08:35.767251 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:08:35.767164    2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 15 05:08:35.767251 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:08:35.767225    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25b6a7f1-e50a-4e38-a0db-06739cd7715e-cert podName:25b6a7f1-e50a-4e38-a0db-06739cd7715e nodeName:}" failed. No retries permitted until 2026-07-15 05:08:36.267185346 +0000 UTC m=+594.434781998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/25b6a7f1-e50a-4e38-a0db-06739cd7715e-cert") pod "llmisvc-controller-manager-777c6bd7d5-g4g9w" (UID: "25b6a7f1-e50a-4e38-a0db-06739cd7715e") : secret "llmisvc-webhook-server-cert" not found
Jul 15 05:08:35.767381 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:08:35.767268    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b47826b-4ca2-431a-a537-e77aa23881f1-cert podName:3b47826b-4ca2-431a-a537-e77aa23881f1 nodeName:}" failed. No retries permitted until 2026-07-15 05:08:36.267254138 +0000 UTC m=+594.434850788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3b47826b-4ca2-431a-a537-e77aa23881f1-cert") pod "kserve-controller-manager-d6f4b65b8-92dw4" (UID: "3b47826b-4ca2-431a-a537-e77aa23881f1") : secret "kserve-webhook-server-cert" not found
Jul 15 05:08:35.780267 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.780246    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtsg\" (UniqueName: \"kubernetes.io/projected/3b47826b-4ca2-431a-a537-e77aa23881f1-kube-api-access-4xtsg\") pod \"kserve-controller-manager-d6f4b65b8-92dw4\" (UID: \"3b47826b-4ca2-431a-a537-e77aa23881f1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:35.783650 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:35.783595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9qcg\" (UniqueName: \"kubernetes.io/projected/25b6a7f1-e50a-4e38-a0db-06739cd7715e-kube-api-access-j9qcg\") pod \"llmisvc-controller-manager-777c6bd7d5-g4g9w\" (UID: \"25b6a7f1-e50a-4e38-a0db-06739cd7715e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:36.271369 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.271335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b47826b-4ca2-431a-a537-e77aa23881f1-cert\") pod \"kserve-controller-manager-d6f4b65b8-92dw4\" (UID: \"3b47826b-4ca2-431a-a537-e77aa23881f1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:36.271369 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.271368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25b6a7f1-e50a-4e38-a0db-06739cd7715e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g4g9w\" (UID: \"25b6a7f1-e50a-4e38-a0db-06739cd7715e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:36.274480 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.274448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b47826b-4ca2-431a-a537-e77aa23881f1-cert\") pod \"kserve-controller-manager-d6f4b65b8-92dw4\" (UID: \"3b47826b-4ca2-431a-a537-e77aa23881f1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:36.274581 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.274498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25b6a7f1-e50a-4e38-a0db-06739cd7715e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g4g9w\" (UID: \"25b6a7f1-e50a-4e38-a0db-06739cd7715e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:36.414829 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.414799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:36.422770 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.422739    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:36.580743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.580714    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"]
Jul 15 05:08:36.599531 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:36.599497    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"]
Jul 15 05:08:36.600948 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:08:36.600921    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod25b6a7f1_e50a_4e38_a0db_06739cd7715e.slice/crio-c9609653817529c56af8692e57c155adf65f048b848933b51a57b7968f0e021e WatchSource:0}: Error finding container c9609653817529c56af8692e57c155adf65f048b848933b51a57b7968f0e021e: Status 404 returned error can't find the container with id c9609653817529c56af8692e57c155adf65f048b848933b51a57b7968f0e021e
Jul 15 05:08:37.482722 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:37.482618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4" event={"ID":"3b47826b-4ca2-431a-a537-e77aa23881f1","Type":"ContainerStarted","Data":"062067aebaace8472abc2138c0bf786e88c6415d3a6e32dc3fe21d2db7621930"}
Jul 15 05:08:37.484483 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:37.484396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w" event={"ID":"25b6a7f1-e50a-4e38-a0db-06739cd7715e","Type":"ContainerStarted","Data":"c9609653817529c56af8692e57c155adf65f048b848933b51a57b7968f0e021e"}
Jul 15 05:08:40.318870 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:40.318840    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk268"]
Jul 15 05:08:40.319296 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:40.319040    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268" podUID="75b950ef-9820-4101-adba-7898cc6ab74c" containerName="discovery" containerID="cri-o://69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b" gracePeriod=30
Jul 15 05:08:41.204646 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.204605    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:08:41.204751 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.204689    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:08:41.204751 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.204739    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:08:41.346474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.346450    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:41.453109 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.453030    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dvhc6"
Jul 15 05:08:41.473648 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.473621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-222x5"
Jul 15 05:08:41.505112 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.505070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g" event={"ID":"d0b7352d-a5ae-4e48-aa10-919032c86c1f","Type":"ContainerStarted","Data":"41abc5b04777917c9aaf95e3a4b10c5c70e36360d94da2dda43b276a5e58cb53"}
Jul 15 05:08:41.507898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.507513    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4" event={"ID":"3b47826b-4ca2-431a-a537-e77aa23881f1","Type":"ContainerStarted","Data":"c2f4b7b533d2be148b2a89869d0cc40d15300edd6c44b14239588449fb26ab30"}
Jul 15 05:08:41.507898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.507576    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:08:41.509447 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.509381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w" event={"ID":"25b6a7f1-e50a-4e38-a0db-06739cd7715e","Type":"ContainerStarted","Data":"f428eae28b0831855a53b1fefa90502a077bfb312d1c3303d2486b9bb20d5556"}
Jul 15 05:08:41.509753 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.509700    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:08:41.511173 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.511146    2581 generic.go:356] "Generic (PLEG): container finished" podID="75b950ef-9820-4101-adba-7898cc6ab74c" containerID="69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b" exitCode=0
Jul 15 05:08:41.511321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.511301    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268"
Jul 15 05:08:41.511382 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.511330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268" event={"ID":"75b950ef-9820-4101-adba-7898cc6ab74c","Type":"ContainerDied","Data":"69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b"}
Jul 15 05:08:41.511382 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.511373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lk268" event={"ID":"75b950ef-9820-4101-adba-7898cc6ab74c","Type":"ContainerDied","Data":"df31669c45b51c2c5cad19b4793cc2b2e938b73bb06047086763d5cafad565aa"}
Jul 15 05:08:41.511446 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.511398    2581 scope.go:117] "RemoveContainer" containerID="69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b"
Jul 15 05:08:41.520442 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.520423    2581 scope.go:117] "RemoveContainer" containerID="69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b"
Jul 15 05:08:41.520893 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:08:41.520871    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b\": container with ID starting with 69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b not found: ID does not exist" containerID="69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b"
Jul 15 05:08:41.521040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.520951    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b"} err="failed to get container status \"69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b\": rpc error: code = NotFound desc = could not find container \"69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b\": container with ID starting with 69e4ded5e358ea138eb8e79e2519a09ece7dc9446b3b195bf0321dc35be93c3b not found: ID does not exist"
Jul 15 05:08:41.525047 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.525025    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bvbn\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-kube-api-access-8bvbn\") pod \"75b950ef-9820-4101-adba-7898cc6ab74c\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") "
Jul 15 05:08:41.525235 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.525191    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-cacerts\") pod \"75b950ef-9820-4101-adba-7898cc6ab74c\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") "
Jul 15 05:08:41.525377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.525362    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b950ef-9820-4101-adba-7898cc6ab74c-local-certs\") pod \"75b950ef-9820-4101-adba-7898cc6ab74c\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") "
Jul 15 05:08:41.525548 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.525532    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-dns-cert\") pod \"75b950ef-9820-4101-adba-7898cc6ab74c\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") "
Jul 15 05:08:41.525686 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.525671    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-istio-token\") pod \"75b950ef-9820-4101-adba-7898cc6ab74c\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") "
Jul 15 05:08:41.525799 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.525785    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-kubeconfig\") pod \"75b950ef-9820-4101-adba-7898cc6ab74c\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") "
Jul 15 05:08:41.525919 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.525905    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-ca-configmap\") pod \"75b950ef-9820-4101-adba-7898cc6ab74c\" (UID: \"75b950ef-9820-4101-adba-7898cc6ab74c\") "
Jul 15 05:08:41.526593 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.526570    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "75b950ef-9820-4101-adba-7898cc6ab74c" (UID: "75b950ef-9820-4101-adba-7898cc6ab74c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:08:41.529337 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.529299    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-cacerts" (OuterVolumeSpecName: "cacerts") pod "75b950ef-9820-4101-adba-7898cc6ab74c" (UID: "75b950ef-9820-4101-adba-7898cc6ab74c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:08:41.530074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.530026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b950ef-9820-4101-adba-7898cc6ab74c-local-certs" (OuterVolumeSpecName: "local-certs") pod "75b950ef-9820-4101-adba-7898cc6ab74c" (UID: "75b950ef-9820-4101-adba-7898cc6ab74c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:08:41.530224 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.530083    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "75b950ef-9820-4101-adba-7898cc6ab74c" (UID: "75b950ef-9820-4101-adba-7898cc6ab74c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:08:41.530513 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.530445    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "75b950ef-9820-4101-adba-7898cc6ab74c" (UID: "75b950ef-9820-4101-adba-7898cc6ab74c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:08:41.530513 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.530451    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-istio-token" (OuterVolumeSpecName: "istio-token") pod "75b950ef-9820-4101-adba-7898cc6ab74c" (UID: "75b950ef-9820-4101-adba-7898cc6ab74c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:08:41.531494 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.531468    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-kube-api-access-8bvbn" (OuterVolumeSpecName: "kube-api-access-8bvbn") pod "75b950ef-9820-4101-adba-7898cc6ab74c" (UID: "75b950ef-9820-4101-adba-7898cc6ab74c"). InnerVolumeSpecName "kube-api-access-8bvbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:08:41.546363 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.546321    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w" podStartSLOduration=1.948638959 podStartE2EDuration="6.546307559s" totalImagesPullingTime="4.5976686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:41.544676496 +0000 UTC m=+599.712273168" watchObservedRunningTime="2026-07-15 05:08:41.546307559 +0000 UTC m=+599.713904224"
Jul 15 05:08:41.546779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.546751    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g" podStartSLOduration=1.640569344 podStartE2EDuration="7.546741316s" totalImagesPullingTime="5.906171972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:41.526682392 +0000 UTC m=+599.694279066" watchObservedRunningTime="2026-07-15 05:08:41.546741316 +0000 UTC m=+599.714337989"
Jul 15 05:08:41.567036 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.566997    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4" podStartSLOduration=1.957454778 podStartE2EDuration="6.566980659s" totalImagesPullingTime="4.609525881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:41.565880138 +0000 UTC m=+599.733476811" watchObservedRunningTime="2026-07-15 05:08:41.566980659 +0000 UTC m=+599.734577373"
Jul 15 05:08:41.627739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.627630    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-dns-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:08:41.627739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.627659    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:08:41.627739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.627676    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-istio-kubeconfig\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:08:41.627739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.627690    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75b950ef-9820-4101-adba-7898cc6ab74c-istio-csr-ca-configmap\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:08:41.627739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.627704    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bvbn\" (UniqueName: \"kubernetes.io/projected/75b950ef-9820-4101-adba-7898cc6ab74c-kube-api-access-8bvbn\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:08:41.627739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.627715    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75b950ef-9820-4101-adba-7898cc6ab74c-cacerts\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:08:41.627739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.627724    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b950ef-9820-4101-adba-7898cc6ab74c-local-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:08:41.833388 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.833359    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk268"]
Jul 15 05:08:41.837083 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:41.837057    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lk268"]
Jul 15 05:08:42.150422 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:42.150343    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:42.154512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:42.154488    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:42.387311 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:42.387282    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b950ef-9820-4101-adba-7898cc6ab74c" path="/var/lib/kubelet/pods/75b950ef-9820-4101-adba-7898cc6ab74c/volumes"
Jul 15 05:08:42.516225 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:42.516136    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:08:42.517013 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:08:42.516988    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s7h8g"
Jul 15 05:09:12.517517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:12.517437    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g4g9w"
Jul 15 05:09:12.587453 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:12.520513    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-92dw4"
Jul 15 05:09:24.360762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.360726    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqrkt"]
Jul 15 05:09:24.361122 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.361058    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b950ef-9820-4101-adba-7898cc6ab74c" containerName="discovery"
Jul 15 05:09:24.361122 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.361069    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b950ef-9820-4101-adba-7898cc6ab74c" containerName="discovery"
Jul 15 05:09:24.361187 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.361131    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b950ef-9820-4101-adba-7898cc6ab74c" containerName="discovery"
Jul 15 05:09:24.363488 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.363471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt"
Jul 15 05:09:24.366290 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.366268    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 05:09:24.366412 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.366342    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 05:09:24.366412 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.366344    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-dnbgs\""
Jul 15 05:09:24.372502 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.372476    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqrkt"]
Jul 15 05:09:24.445773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.445746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcb5z\" (UniqueName: \"kubernetes.io/projected/e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15-kube-api-access-vcb5z\") pod \"kuadrant-operator-catalog-fqrkt\" (UID: \"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15\") " pod="kuadrant-system/kuadrant-operator-catalog-fqrkt"
Jul 15 05:09:24.546760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.546729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcb5z\" (UniqueName: \"kubernetes.io/projected/e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15-kube-api-access-vcb5z\") pod \"kuadrant-operator-catalog-fqrkt\" (UID: \"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15\") " pod="kuadrant-system/kuadrant-operator-catalog-fqrkt"
Jul 15 05:09:24.555055 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.555027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcb5z\" (UniqueName: \"kubernetes.io/projected/e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15-kube-api-access-vcb5z\") pod \"kuadrant-operator-catalog-fqrkt\" (UID: \"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15\") " pod="kuadrant-system/kuadrant-operator-catalog-fqrkt"
Jul 15 05:09:24.672910 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.672858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt"
Jul 15 05:09:24.726952 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.726920    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqrkt"]
Jul 15 05:09:24.803531 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.803509    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqrkt"]
Jul 15 05:09:24.806119 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:09:24.806088    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c4fd60_7a34_4b8d_8f50_7e6dbcd2ec15.slice/crio-d1a7c9b43cb77c2e2b9ff06260fa411716b31c7acfd37e327ec291e0d86856be WatchSource:0}: Error finding container d1a7c9b43cb77c2e2b9ff06260fa411716b31c7acfd37e327ec291e0d86856be: Status 404 returned error can't find the container with id d1a7c9b43cb77c2e2b9ff06260fa411716b31c7acfd37e327ec291e0d86856be
Jul 15 05:09:24.807638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.807615    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:09:24.937935 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.937875    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dz279"]
Jul 15 05:09:24.941178 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.941162    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:24.947938 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:24.947915    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dz279"]
Jul 15 05:09:25.050810 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:25.050787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4zmd\" (UniqueName: \"kubernetes.io/projected/6fd60195-c7cb-4386-8e2b-2fb929fe41e8-kube-api-access-r4zmd\") pod \"kuadrant-operator-catalog-dz279\" (UID: \"6fd60195-c7cb-4386-8e2b-2fb929fe41e8\") " pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:25.151344 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:25.151316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4zmd\" (UniqueName: \"kubernetes.io/projected/6fd60195-c7cb-4386-8e2b-2fb929fe41e8-kube-api-access-r4zmd\") pod \"kuadrant-operator-catalog-dz279\" (UID: \"6fd60195-c7cb-4386-8e2b-2fb929fe41e8\") " pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:25.159572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:25.159542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4zmd\" (UniqueName: \"kubernetes.io/projected/6fd60195-c7cb-4386-8e2b-2fb929fe41e8-kube-api-access-r4zmd\") pod \"kuadrant-operator-catalog-dz279\" (UID: \"6fd60195-c7cb-4386-8e2b-2fb929fe41e8\") " pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:25.250643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:25.250577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:25.379329 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:25.379306    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dz279"]
Jul 15 05:09:25.384007 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:09:25.383978    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd60195_c7cb_4386_8e2b_2fb929fe41e8.slice/crio-412cbebd960cd8601a1220247f01e4b0328aa56eb6c070ac0c7e07e6ca721de7 WatchSource:0}: Error finding container 412cbebd960cd8601a1220247f01e4b0328aa56eb6c070ac0c7e07e6ca721de7: Status 404 returned error can't find the container with id 412cbebd960cd8601a1220247f01e4b0328aa56eb6c070ac0c7e07e6ca721de7
Jul 15 05:09:25.699712 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:25.699673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt" event={"ID":"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15","Type":"ContainerStarted","Data":"d1a7c9b43cb77c2e2b9ff06260fa411716b31c7acfd37e327ec291e0d86856be"}
Jul 15 05:09:25.700815 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:25.700782    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dz279" event={"ID":"6fd60195-c7cb-4386-8e2b-2fb929fe41e8","Type":"ContainerStarted","Data":"412cbebd960cd8601a1220247f01e4b0328aa56eb6c070ac0c7e07e6ca721de7"}
Jul 15 05:09:27.711292 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:27.711259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt" event={"ID":"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15","Type":"ContainerStarted","Data":"82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9"}
Jul 15 05:09:27.711741 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:27.711320    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt" podUID="e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15" containerName="registry-server" containerID="cri-o://82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9" gracePeriod=2
Jul 15 05:09:27.712920 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:27.712896    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dz279" event={"ID":"6fd60195-c7cb-4386-8e2b-2fb929fe41e8","Type":"ContainerStarted","Data":"5c1c0e39f16d6faa2921bd49c24d8800753f855e8ae59165a3a7e953754b555a"}
Jul 15 05:09:27.726341 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:27.726302    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt" podStartSLOduration=1.6068573320000001 podStartE2EDuration="3.726287874s" totalImagesPullingTime="2.119430542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:09:27.72577044 +0000 UTC m=+645.893367140" watchObservedRunningTime="2026-07-15 05:09:27.726287874 +0000 UTC m=+645.893884538"
Jul 15 05:09:27.741269 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:27.741232    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dz279" podStartSLOduration=2.187933575 podStartE2EDuration="3.741219761s" totalImagesPullingTime="1.553286186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:09:27.739355196 +0000 UTC m=+645.906951868" watchObservedRunningTime="2026-07-15 05:09:27.741219761 +0000 UTC m=+645.908816430"
Jul 15 05:09:27.955074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:27.955051    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt"
Jul 15 05:09:28.077040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.077019    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcb5z\" (UniqueName: \"kubernetes.io/projected/e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15-kube-api-access-vcb5z\") pod \"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15\" (UID: \"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15\") "
Jul 15 05:09:28.079121 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.079090    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15-kube-api-access-vcb5z" (OuterVolumeSpecName: "kube-api-access-vcb5z") pod "e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15" (UID: "e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15"). InnerVolumeSpecName "kube-api-access-vcb5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:09:28.178428 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.178402    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcb5z\" (UniqueName: \"kubernetes.io/projected/e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15-kube-api-access-vcb5z\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:09:28.717928 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.717858    2581 generic.go:356] "Generic (PLEG): container finished" podID="e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15" containerID="82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9" exitCode=0
Jul 15 05:09:28.717928 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.717901    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt"
Jul 15 05:09:28.718390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.717940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt" event={"ID":"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15","Type":"ContainerDied","Data":"82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9"}
Jul 15 05:09:28.718390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.717996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqrkt" event={"ID":"e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15","Type":"ContainerDied","Data":"d1a7c9b43cb77c2e2b9ff06260fa411716b31c7acfd37e327ec291e0d86856be"}
Jul 15 05:09:28.718390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.718023    2581 scope.go:117] "RemoveContainer" containerID="82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9"
Jul 15 05:09:28.726413 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.726396    2581 scope.go:117] "RemoveContainer" containerID="82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9"
Jul 15 05:09:28.726649 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:09:28.726628    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9\": container with ID starting with 82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9 not found: ID does not exist" containerID="82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9"
Jul 15 05:09:28.726713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.726657    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9"} err="failed to get container status \"82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9\": rpc error: code = NotFound desc = could not find container \"82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9\": container with ID starting with 82127e1059406cf27fd285eb0509fcb5f62ddfd4dfd99b42e46283b179411ba9 not found: ID does not exist"
Jul 15 05:09:28.757412 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.757381    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqrkt"]
Jul 15 05:09:28.761985 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:28.761964    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqrkt"]
Jul 15 05:09:30.387582 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:30.387552    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15" path="/var/lib/kubelet/pods/e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15/volumes"
Jul 15 05:09:35.251070 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:35.251034    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:35.251469 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:35.251086    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:35.273047 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:35.273023    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:35.771851 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:35.771823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dz279"
Jul 15 05:09:57.041687 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.041654    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"]
Jul 15 05:09:57.042133 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.041960    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15" containerName="registry-server"
Jul 15 05:09:57.042133 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.041973    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15" containerName="registry-server"
Jul 15 05:09:57.042133 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.042032    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c4fd60-7a34-4b8d-8f50-7e6dbcd2ec15" containerName="registry-server"
Jul 15 05:09:57.046477 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.046457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:09:57.051032 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.051008    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qr7fq\""
Jul 15 05:09:57.055679 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.055656    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"]
Jul 15 05:09:57.195355 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.195320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7cjm\" (UniqueName: \"kubernetes.io/projected/857291a5-10cd-414f-8405-a9a7cf00e2f4-kube-api-access-d7cjm\") pod \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" (UID: \"857291a5-10cd-414f-8405-a9a7cf00e2f4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:09:57.296256 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.296176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7cjm\" (UniqueName: \"kubernetes.io/projected/857291a5-10cd-414f-8405-a9a7cf00e2f4-kube-api-access-d7cjm\") pod \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" (UID: \"857291a5-10cd-414f-8405-a9a7cf00e2f4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:09:57.306412 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.306386    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7cjm\" (UniqueName: \"kubernetes.io/projected/857291a5-10cd-414f-8405-a9a7cf00e2f4-kube-api-access-d7cjm\") pod \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" (UID: \"857291a5-10cd-414f-8405-a9a7cf00e2f4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:09:57.356786 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.356748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:09:57.482389 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.482366    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"]
Jul 15 05:09:57.484272 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:09:57.484244    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857291a5_10cd_414f_8405_a9a7cf00e2f4.slice/crio-b41735bb9f4fb9f83c99f023f8c5540e38938e0216fd81787dd0c978ca15036a WatchSource:0}: Error finding container b41735bb9f4fb9f83c99f023f8c5540e38938e0216fd81787dd0c978ca15036a: Status 404 returned error can't find the container with id b41735bb9f4fb9f83c99f023f8c5540e38938e0216fd81787dd0c978ca15036a
Jul 15 05:09:57.842541 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:57.842508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq" event={"ID":"857291a5-10cd-414f-8405-a9a7cf00e2f4","Type":"ContainerStarted","Data":"b41735bb9f4fb9f83c99f023f8c5540e38938e0216fd81787dd0c978ca15036a"}
Jul 15 05:09:59.853590 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:59.853556    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq" event={"ID":"857291a5-10cd-414f-8405-a9a7cf00e2f4","Type":"ContainerStarted","Data":"f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3"}
Jul 15 05:09:59.853960 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:59.853672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:09:59.877165 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:09:59.877119    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq" podStartSLOduration=0.957484331 podStartE2EDuration="2.877101589s" totalImagesPullingTime="1.919617258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:09:59.875327743 +0000 UTC m=+678.042924426" watchObservedRunningTime="2026-07-15 05:09:59.877101589 +0000 UTC m=+678.044698260"
Jul 15 05:10:02.325727 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.325687    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"]
Jul 15 05:10:02.329293 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.329268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.332850 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.332830    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p4hm7\""
Jul 15 05:10:02.341521 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.341495    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.341604 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.341543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr4sp\" (UniqueName: \"kubernetes.io/projected/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-kube-api-access-tr4sp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.346459 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.346439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"]
Jul 15 05:10:02.442021 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.441987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.442234 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.442055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr4sp\" (UniqueName: \"kubernetes.io/projected/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-kube-api-access-tr4sp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.442491 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.442469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.454420 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.454394    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr4sp\" (UniqueName: \"kubernetes.io/projected/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-kube-api-access-tr4sp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.638865 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.638787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:02.773772 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.773745    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"]
Jul 15 05:10:02.776803 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:02.776780    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b4e0d7_8969_4b6d_86cc_d984dd85e587.slice/crio-d15def2c926f5e3d1b976eee6e54d3aa82388202672fe145d00e9babf631c145 WatchSource:0}: Error finding container d15def2c926f5e3d1b976eee6e54d3aa82388202672fe145d00e9babf631c145: Status 404 returned error can't find the container with id d15def2c926f5e3d1b976eee6e54d3aa82388202672fe145d00e9babf631c145
Jul 15 05:10:02.867145 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:02.867115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp" event={"ID":"d7b4e0d7-8969-4b6d-86cc-d984dd85e587","Type":"ContainerStarted","Data":"d15def2c926f5e3d1b976eee6e54d3aa82388202672fe145d00e9babf631c145"}
Jul 15 05:10:04.967875 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:04.967843    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"]
Jul 15 05:10:04.970482 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:04.970466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"
Jul 15 05:10:04.973367 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:04.973348    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-w5lwg\""
Jul 15 05:10:04.980058 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:04.980038    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"]
Jul 15 05:10:05.065539 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:05.065508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smcdz\" (UniqueName: \"kubernetes.io/projected/f49a82f8-2896-49b7-999f-9cc93e98f207-kube-api-access-smcdz\") pod \"authorino-operator-75cc8d5cb7-tfvm2\" (UID: \"f49a82f8-2896-49b7-999f-9cc93e98f207\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"
Jul 15 05:10:05.166905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:05.166881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smcdz\" (UniqueName: \"kubernetes.io/projected/f49a82f8-2896-49b7-999f-9cc93e98f207-kube-api-access-smcdz\") pod \"authorino-operator-75cc8d5cb7-tfvm2\" (UID: \"f49a82f8-2896-49b7-999f-9cc93e98f207\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"
Jul 15 05:10:05.187059 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:05.187033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smcdz\" (UniqueName: \"kubernetes.io/projected/f49a82f8-2896-49b7-999f-9cc93e98f207-kube-api-access-smcdz\") pod \"authorino-operator-75cc8d5cb7-tfvm2\" (UID: \"f49a82f8-2896-49b7-999f-9cc93e98f207\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"
Jul 15 05:10:05.281056 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:05.281001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"
Jul 15 05:10:05.416541 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:05.416499    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"]
Jul 15 05:10:05.418172 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:05.418136    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf49a82f8_2896_49b7_999f_9cc93e98f207.slice/crio-3a6b8868a60539ccf8fb69fd04229b5824757fb70a2b453b0ad9f89d1066a65e WatchSource:0}: Error finding container 3a6b8868a60539ccf8fb69fd04229b5824757fb70a2b453b0ad9f89d1066a65e: Status 404 returned error can't find the container with id 3a6b8868a60539ccf8fb69fd04229b5824757fb70a2b453b0ad9f89d1066a65e
Jul 15 05:10:05.880068 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:05.880033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2" event={"ID":"f49a82f8-2896-49b7-999f-9cc93e98f207","Type":"ContainerStarted","Data":"3a6b8868a60539ccf8fb69fd04229b5824757fb70a2b453b0ad9f89d1066a65e"}
Jul 15 05:10:07.893221 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:07.893124    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2" event={"ID":"f49a82f8-2896-49b7-999f-9cc93e98f207","Type":"ContainerStarted","Data":"ad2958014c9668a315846f6e68f44bc0c1af86bf396d5d5deb8afa66da83dc5f"}
Jul 15 05:10:07.893567 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:07.893245    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"
Jul 15 05:10:07.910889 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:07.910850    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2" podStartSLOduration=1.748093612 podStartE2EDuration="3.910836543s" totalImagesPullingTime="2.162742931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:07.909688672 +0000 UTC m=+686.077285349" watchObservedRunningTime="2026-07-15 05:10:07.910836543 +0000 UTC m=+686.078433215"
Jul 15 05:10:10.859760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:10.859731    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:10.908821 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:10.908793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp" event={"ID":"d7b4e0d7-8969-4b6d-86cc-d984dd85e587","Type":"ContainerStarted","Data":"137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2"}
Jul 15 05:10:10.908963 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:10.908862    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:10.928411 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:10.928354    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp" podStartSLOduration=1.625373247 podStartE2EDuration="8.928324751s" totalImagesPullingTime="7.302951504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:10.926236511 +0000 UTC m=+689.093833186" watchObservedRunningTime="2026-07-15 05:10:10.928324751 +0000 UTC m=+689.095921424"
Jul 15 05:10:18.900267 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:18.900226    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tfvm2"
Jul 15 05:10:21.915315 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:21.915285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:22.965133 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:22.965100    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"]
Jul 15 05:10:22.967680 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:22.967659    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:22.991195 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:22.991168    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"]
Jul 15 05:10:23.007753 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.007727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv42h\" (UniqueName: \"kubernetes.io/projected/6b3332de-413b-4039-ba8a-da40431ff3d2-kube-api-access-fv42h\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.007869 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.007828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b3332de-413b-4039-ba8a-da40431ff3d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.108888 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.108860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b3332de-413b-4039-ba8a-da40431ff3d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.109010 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.108913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv42h\" (UniqueName: \"kubernetes.io/projected/6b3332de-413b-4039-ba8a-da40431ff3d2-kube-api-access-fv42h\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.109316 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.109296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b3332de-413b-4039-ba8a-da40431ff3d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.123868 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.123844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv42h\" (UniqueName: \"kubernetes.io/projected/6b3332de-413b-4039-ba8a-da40431ff3d2-kube-api-access-fv42h\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.278286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.278214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.412272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.412221    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"]
Jul 15 05:10:23.414780 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:23.414755    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3332de_413b_4039_ba8a_da40431ff3d2.slice/crio-bd063a44b7a0d44ac662e7cc7e01f0eb7ba4c79e81916b2b6797b0fdddf2a1b4 WatchSource:0}: Error finding container bd063a44b7a0d44ac662e7cc7e01f0eb7ba4c79e81916b2b6797b0fdddf2a1b4: Status 404 returned error can't find the container with id bd063a44b7a0d44ac662e7cc7e01f0eb7ba4c79e81916b2b6797b0fdddf2a1b4
Jul 15 05:10:23.690915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.690885    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"]
Jul 15 05:10:23.697173 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.697147    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"]
Jul 15 05:10:23.709563 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.709537    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"]
Jul 15 05:10:23.709793 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.709753    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" containerName="manager" containerID="cri-o://137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2" gracePeriod=2
Jul 15 05:10:23.713514 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.713490    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"]
Jul 15 05:10:23.714098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.714054    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" containerName="manager"
Jul 15 05:10:23.714098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.714082    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" containerName="manager"
Jul 15 05:10:23.714274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.714237    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" containerName="manager"
Jul 15 05:10:23.717520 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.717498    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"]
Jul 15 05:10:23.717631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.717602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:23.720843 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.720811    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.730531 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.730507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"]
Jul 15 05:10:23.732530 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.732508    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"]
Jul 15 05:10:23.732715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.732694    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" containerName="manager" containerID="cri-o://f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3" gracePeriod=2
Jul 15 05:10:23.734295 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.734269    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"]
Jul 15 05:10:23.734600 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.734588    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" containerName="manager"
Jul 15 05:10:23.734638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.734603    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" containerName="manager"
Jul 15 05:10:23.734676 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.734666    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" containerName="manager"
Jul 15 05:10:23.737307 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.737284    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:23.743951 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.743932    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"]
Jul 15 05:10:23.762604 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.762583    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"]
Jul 15 05:10:23.793349 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.793318    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:23.795407 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.795385    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.814969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.814947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4lxv\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:23.815074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.814989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9bc8\" (UniqueName: \"kubernetes.io/projected/b40ce13d-5697-4be2-b146-a0cb1342df91-kube-api-access-s9bc8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-n5vxr\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:23.815074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.815014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ghkd\" (UniqueName: \"kubernetes.io/projected/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-kube-api-access-7ghkd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4lxv\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:23.815074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.815049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b40ce13d-5697-4be2-b146-a0cb1342df91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-n5vxr\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:23.915732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.915699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9bc8\" (UniqueName: \"kubernetes.io/projected/b40ce13d-5697-4be2-b146-a0cb1342df91-kube-api-access-s9bc8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-n5vxr\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:23.915864 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.915742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ghkd\" (UniqueName: \"kubernetes.io/projected/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-kube-api-access-7ghkd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4lxv\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:23.915864 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.915767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b40ce13d-5697-4be2-b146-a0cb1342df91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-n5vxr\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:23.916004 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.915875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4lxv\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:23.916306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.916278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b40ce13d-5697-4be2-b146-a0cb1342df91-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-n5vxr\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:23.916405 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.916286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4lxv\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:23.928306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.928281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9bc8\" (UniqueName: \"kubernetes.io/projected/b40ce13d-5697-4be2-b146-a0cb1342df91-kube-api-access-s9bc8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-n5vxr\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:23.928426 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.928367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ghkd\" (UniqueName: \"kubernetes.io/projected/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-kube-api-access-7ghkd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j4lxv\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:23.956301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.956280    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.958533 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.958500    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:23.960524 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.960496    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.962547 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.962529    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:23.964780 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.964754    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:23.966720 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.966697    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.977256 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.977192    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s" container="manager"
Jul 15 05:10:23.979248 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.979167    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" containerID="137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2" exitCode=0
Jul 15 05:10:23.979393 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.979316    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.979572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.979322    2581 scope.go:117] "RemoveContainer" containerID="137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2"
Jul 15 05:10:23.979850 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.979789    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.980799 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.980776    2581 generic.go:356] "Generic (PLEG): container finished" podID="857291a5-10cd-414f-8405-a9a7cf00e2f4" containerID="f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3" exitCode=0
Jul 15 05:10:23.980937 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.980812    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:23.981780 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.981753    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:23.983653 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.983630    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.985806 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.985784    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:23.987612 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.987586    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:23.988827 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.988808    2581 scope.go:117] "RemoveContainer" containerID="137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2"
Jul 15 05:10:23.989356 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.989331    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2\": container with ID starting with 137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2 not found: ID does not exist" containerID="137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2"
Jul 15 05:10:23.989452 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.989363    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2"} err="failed to get container status \"137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2\": rpc error: code = NotFound desc = could not find container \"137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2\": container with ID starting with 137eacae119ee34e02768f51ff87e066f71cd649ba8fd3ee76ca9ba998d8faf2 not found: ID does not exist"
Jul 15 05:10:23.989452 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.989379    2581 scope.go:117] "RemoveContainer" containerID="f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3"
Jul 15 05:10:23.989628 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.989605    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:23.998719 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.998698    2581 scope.go:117] "RemoveContainer" containerID="f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3"
Jul 15 05:10:23.998971 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:23.998954    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3\": container with ID starting with f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3 not found: ID does not exist" containerID="f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3"
Jul 15 05:10:23.999019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:23.998977    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3"} err="failed to get container status \"f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3\": rpc error: code = NotFound desc = could not find container \"f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3\": container with ID starting with f801aeb6fcdea8353ddac4e79938c7063dcd30f6914a16d5b5198769939b6cf3 not found: ID does not exist"
Jul 15 05:10:24.016740 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.016336    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7cjm\" (UniqueName: \"kubernetes.io/projected/857291a5-10cd-414f-8405-a9a7cf00e2f4-kube-api-access-d7cjm\") pod \"857291a5-10cd-414f-8405-a9a7cf00e2f4\" (UID: \"857291a5-10cd-414f-8405-a9a7cf00e2f4\") "
Jul 15 05:10:24.016740 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.016395    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tr4sp\" (UniqueName: \"kubernetes.io/projected/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-kube-api-access-tr4sp\") pod \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") "
Jul 15 05:10:24.016740 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.016494    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-extensions-socket-volume\") pod \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\" (UID: \"d7b4e0d7-8969-4b6d-86cc-d984dd85e587\") "
Jul 15 05:10:24.017255 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.017227    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d7b4e0d7-8969-4b6d-86cc-d984dd85e587" (UID: "d7b4e0d7-8969-4b6d-86cc-d984dd85e587"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:10:24.018586 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.018564    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-kube-api-access-tr4sp" (OuterVolumeSpecName: "kube-api-access-tr4sp") pod "d7b4e0d7-8969-4b6d-86cc-d984dd85e587" (UID: "d7b4e0d7-8969-4b6d-86cc-d984dd85e587"). InnerVolumeSpecName "kube-api-access-tr4sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:10:24.018660 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.018582    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857291a5-10cd-414f-8405-a9a7cf00e2f4-kube-api-access-d7cjm" (OuterVolumeSpecName: "kube-api-access-d7cjm") pod "857291a5-10cd-414f-8405-a9a7cf00e2f4" (UID: "857291a5-10cd-414f-8405-a9a7cf00e2f4"). InnerVolumeSpecName "kube-api-access-d7cjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:10:24.031528 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.031509    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:24.117809 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.117781    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-extensions-socket-volume\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:24.117809 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.117808    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7cjm\" (UniqueName: \"kubernetes.io/projected/857291a5-10cd-414f-8405-a9a7cf00e2f4-kube-api-access-d7cjm\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:24.117995 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.117818    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr4sp\" (UniqueName: \"kubernetes.io/projected/d7b4e0d7-8969-4b6d-86cc-d984dd85e587-kube-api-access-tr4sp\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:24.140773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.140727    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:24.167430 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.167407    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"]
Jul 15 05:10:24.169810 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:24.169788    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40ce13d_5697_4be2_b146_a0cb1342df91.slice/crio-c3b2483f5d0a4120f815117d9f65363cbed1ec1d21a59821f589eac298f1310c WatchSource:0}: Error finding container c3b2483f5d0a4120f815117d9f65363cbed1ec1d21a59821f589eac298f1310c: Status 404 returned error can't find the container with id c3b2483f5d0a4120f815117d9f65363cbed1ec1d21a59821f589eac298f1310c
Jul 15 05:10:24.278905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.278880    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"]
Jul 15 05:10:24.281298 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:24.281260    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cd258c_b6aa_4ec4_94ff_e0611a1ac278.slice/crio-edbfeace802291c592dd35d879f02940d54068199170029f4824328cef221795 WatchSource:0}: Error finding container edbfeace802291c592dd35d879f02940d54068199170029f4824328cef221795: Status 404 returned error can't find the container with id edbfeace802291c592dd35d879f02940d54068199170029f4824328cef221795
Jul 15 05:10:24.293464 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:24.293437    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:24.295383 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:24.295356    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:24.297249 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:24.297191    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:24.299235 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:24.299198    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-96vgp\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-96vgp"
Jul 15 05:10:24.301289 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:24.301268    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:24.303170 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:24.303148    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dqgtq\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dqgtq"
Jul 15 05:10:24.393761 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.393729    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" path="/var/lib/kubelet/pods/857291a5-10cd-414f-8405-a9a7cf00e2f4/volumes"
Jul 15 05:10:24.394235 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.394192    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7b4e0d7-8969-4b6d-86cc-d984dd85e587" path="/var/lib/kubelet/pods/d7b4e0d7-8969-4b6d-86cc-d984dd85e587/volumes"
Jul 15 05:10:24.988998 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.988958    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv" event={"ID":"89cd258c-b6aa-4ec4-94ff-e0611a1ac278","Type":"ContainerStarted","Data":"66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794"}
Jul 15 05:10:24.989429 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.989017    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv" event={"ID":"89cd258c-b6aa-4ec4-94ff-e0611a1ac278","Type":"ContainerStarted","Data":"edbfeace802291c592dd35d879f02940d54068199170029f4824328cef221795"}
Jul 15 05:10:24.989429 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.989071    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:24.990889 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.990865    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr" event={"ID":"b40ce13d-5697-4be2-b146-a0cb1342df91","Type":"ContainerStarted","Data":"353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1"}
Jul 15 05:10:24.991011 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.990913    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:24.991011 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.990934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr" event={"ID":"b40ce13d-5697-4be2-b146-a0cb1342df91","Type":"ContainerStarted","Data":"c3b2483f5d0a4120f815117d9f65363cbed1ec1d21a59821f589eac298f1310c"}
Jul 15 05:10:24.992461 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:24.992427    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" containerName="manager" containerID="cri-o://edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f" gracePeriod=2
Jul 15 05:10:25.013455 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.013415    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv" podStartSLOduration=2.013402069 podStartE2EDuration="2.013402069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:25.011925813 +0000 UTC m=+703.179522489" watchObservedRunningTime="2026-07-15 05:10:25.013402069 +0000 UTC m=+703.180998741"
Jul 15 05:10:25.035908 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.035864    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr" podStartSLOduration=2.035851257 podStartE2EDuration="2.035851257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:25.034326513 +0000 UTC m=+703.201923185" watchObservedRunningTime="2026-07-15 05:10:25.035851257 +0000 UTC m=+703.203447995"
Jul 15 05:10:25.234498 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.234475    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:25.236745 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:25.236721    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:25.329215 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.329186    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv42h\" (UniqueName: \"kubernetes.io/projected/6b3332de-413b-4039-ba8a-da40431ff3d2-kube-api-access-fv42h\") pod \"6b3332de-413b-4039-ba8a-da40431ff3d2\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") "
Jul 15 05:10:25.329346 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.329282    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b3332de-413b-4039-ba8a-da40431ff3d2-extensions-socket-volume\") pod \"6b3332de-413b-4039-ba8a-da40431ff3d2\" (UID: \"6b3332de-413b-4039-ba8a-da40431ff3d2\") "
Jul 15 05:10:25.329636 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.329615    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3332de-413b-4039-ba8a-da40431ff3d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6b3332de-413b-4039-ba8a-da40431ff3d2" (UID: "6b3332de-413b-4039-ba8a-da40431ff3d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:10:25.331124 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.331105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3332de-413b-4039-ba8a-da40431ff3d2-kube-api-access-fv42h" (OuterVolumeSpecName: "kube-api-access-fv42h") pod "6b3332de-413b-4039-ba8a-da40431ff3d2" (UID: "6b3332de-413b-4039-ba8a-da40431ff3d2"). InnerVolumeSpecName "kube-api-access-fv42h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:10:25.429872 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.429849    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b3332de-413b-4039-ba8a-da40431ff3d2-extensions-socket-volume\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:25.429872 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.429870    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv42h\" (UniqueName: \"kubernetes.io/projected/6b3332de-413b-4039-ba8a-da40431ff3d2-kube-api-access-fv42h\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:25.999812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.999777    2581 generic.go:356] "Generic (PLEG): container finished" podID="6b3332de-413b-4039-ba8a-da40431ff3d2" containerID="edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f" exitCode=0
Jul 15 05:10:26.000258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.999818    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:26.000258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:25.999873    2581 scope.go:117] "RemoveContainer" containerID="edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f"
Jul 15 05:10:26.002144 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:26.002115    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:26.008308 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:26.008294    2581 scope.go:117] "RemoveContainer" containerID="edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f"
Jul 15 05:10:26.008557 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:26.008539    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f\": container with ID starting with edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f not found: ID does not exist" containerID="edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f"
Jul 15 05:10:26.008602 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:26.008565    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f"} err="failed to get container status \"edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f\": rpc error: code = NotFound desc = could not find container \"edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f\": container with ID starting with edb835509ba0a54693e98c824287ac673f39ddb4abcba4f8fe6440850cef5d7f not found: ID does not exist"
Jul 15 05:10:26.010025 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:26.010006    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rbv4s\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rbv4s"
Jul 15 05:10:26.388540 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:26.388510    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b3332de-413b-4039-ba8a-da40431ff3d2" path="/var/lib/kubelet/pods/6b3332de-413b-4039-ba8a-da40431ff3d2/volumes"
Jul 15 05:10:30.320606 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.318017    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9949b5ff-qnhzh"]
Jul 15 05:10:30.320606 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.318803    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" containerName="manager"
Jul 15 05:10:30.320606 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.318825    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" containerName="manager"
Jul 15 05:10:30.320606 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.318994    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="857291a5-10cd-414f-8405-a9a7cf00e2f4" containerName="manager"
Jul 15 05:10:30.323323 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.323298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.334357 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.334337    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9949b5ff-qnhzh"]
Jul 15 05:10:30.468477 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.468447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-service-ca\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.468614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.468483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-oauth-serving-cert\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.468614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.468573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-oauth-config\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.468731 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.468638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-config\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.468731 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.468670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6tcr\" (UniqueName: \"kubernetes.io/projected/3babc8bd-ac5b-4f00-8c00-193f5784eae6-kube-api-access-s6tcr\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.468812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.468738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-serving-cert\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.468812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.468770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-trusted-ca-bundle\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.569169 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.569143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-config\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.569286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.569171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6tcr\" (UniqueName: \"kubernetes.io/projected/3babc8bd-ac5b-4f00-8c00-193f5784eae6-kube-api-access-s6tcr\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.569286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.569219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-serving-cert\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.569286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.569241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-trusted-ca-bundle\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.569286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.569262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-service-ca\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.569286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.569278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-oauth-serving-cert\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.569514 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.569307    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-oauth-config\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.570123 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.570099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-config\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.570248 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.570099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-service-ca\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.570248 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.570140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-trusted-ca-bundle\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.570248 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.570140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3babc8bd-ac5b-4f00-8c00-193f5784eae6-oauth-serving-cert\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.571769 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.571714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-oauth-config\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.571869 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.571818    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3babc8bd-ac5b-4f00-8c00-193f5784eae6-console-serving-cert\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.577362 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.577342    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6tcr\" (UniqueName: \"kubernetes.io/projected/3babc8bd-ac5b-4f00-8c00-193f5784eae6-kube-api-access-s6tcr\") pod \"console-c9949b5ff-qnhzh\" (UID: \"3babc8bd-ac5b-4f00-8c00-193f5784eae6\") " pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.631952 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.631928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:30.762869 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:30.762846    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9949b5ff-qnhzh"]
Jul 15 05:10:30.765587 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:30.765560    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3babc8bd_ac5b_4f00_8c00_193f5784eae6.slice/crio-aab356d2ee2fbe4852186c07920cc7c850e9a563c86f0b4fd6f5181bed2c451c WatchSource:0}: Error finding container aab356d2ee2fbe4852186c07920cc7c850e9a563c86f0b4fd6f5181bed2c451c: Status 404 returned error can't find the container with id aab356d2ee2fbe4852186c07920cc7c850e9a563c86f0b4fd6f5181bed2c451c
Jul 15 05:10:31.025994 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:31.025957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9949b5ff-qnhzh" event={"ID":"3babc8bd-ac5b-4f00-8c00-193f5784eae6","Type":"ContainerStarted","Data":"7cf887f449d7f16d94afae443191780fa3029edd2070a9af41f6705fd1f17a0c"}
Jul 15 05:10:31.026159 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:31.026013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9949b5ff-qnhzh" event={"ID":"3babc8bd-ac5b-4f00-8c00-193f5784eae6","Type":"ContainerStarted","Data":"aab356d2ee2fbe4852186c07920cc7c850e9a563c86f0b4fd6f5181bed2c451c"}
Jul 15 05:10:31.049997 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:31.049953    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9949b5ff-qnhzh" podStartSLOduration=1.049939157 podStartE2EDuration="1.049939157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:31.047851071 +0000 UTC m=+709.215447743" watchObservedRunningTime="2026-07-15 05:10:31.049939157 +0000 UTC m=+709.217535828"
Jul 15 05:10:36.002847 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.002770    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:36.003285 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.002947    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:36.074828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.074795    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"]
Jul 15 05:10:36.075004 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.074969    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv" podUID="89cd258c-b6aa-4ec4-94ff-e0611a1ac278" containerName="manager" containerID="cri-o://66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794" gracePeriod=10
Jul 15 05:10:36.324257 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.324231    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"]
Jul 15 05:10:36.327087 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.327065    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.330074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.330051    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:36.348552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.348527    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"]
Jul 15 05:10:36.418573 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.418547    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ghkd\" (UniqueName: \"kubernetes.io/projected/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-kube-api-access-7ghkd\") pod \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") "
Jul 15 05:10:36.418698 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.418608    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-extensions-socket-volume\") pod \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\" (UID: \"89cd258c-b6aa-4ec4-94ff-e0611a1ac278\") "
Jul 15 05:10:36.418698 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.418671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00b909e7-1ddd-4078-8da2-3fccf0ab7209-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d6v2p\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.418781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.418741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx8cx\" (UniqueName: \"kubernetes.io/projected/00b909e7-1ddd-4078-8da2-3fccf0ab7209-kube-api-access-kx8cx\") pod \"kuadrant-operator-controller-manager-86878c769f-d6v2p\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.418974 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.418953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "89cd258c-b6aa-4ec4-94ff-e0611a1ac278" (UID: "89cd258c-b6aa-4ec4-94ff-e0611a1ac278"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:10:36.420506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.420479    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-kube-api-access-7ghkd" (OuterVolumeSpecName: "kube-api-access-7ghkd") pod "89cd258c-b6aa-4ec4-94ff-e0611a1ac278" (UID: "89cd258c-b6aa-4ec4-94ff-e0611a1ac278"). InnerVolumeSpecName "kube-api-access-7ghkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:10:36.519182 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.519153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx8cx\" (UniqueName: \"kubernetes.io/projected/00b909e7-1ddd-4078-8da2-3fccf0ab7209-kube-api-access-kx8cx\") pod \"kuadrant-operator-controller-manager-86878c769f-d6v2p\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.519313 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.519236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00b909e7-1ddd-4078-8da2-3fccf0ab7209-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d6v2p\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.519365 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.519318    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ghkd\" (UniqueName: \"kubernetes.io/projected/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-kube-api-access-7ghkd\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:36.519365 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.519333    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89cd258c-b6aa-4ec4-94ff-e0611a1ac278-extensions-socket-volume\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:36.519572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.519551    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00b909e7-1ddd-4078-8da2-3fccf0ab7209-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d6v2p\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.530945 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.530903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx8cx\" (UniqueName: \"kubernetes.io/projected/00b909e7-1ddd-4078-8da2-3fccf0ab7209-kube-api-access-kx8cx\") pod \"kuadrant-operator-controller-manager-86878c769f-d6v2p\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.636790 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.636764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:36.766903 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:36.766776    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"]
Jul 15 05:10:36.769727 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:36.769704    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b909e7_1ddd_4078_8da2_3fccf0ab7209.slice/crio-2b9301feac7e7c08023a4f3a1ce2b786041992ffbaa1b6e547349092b91f29ef WatchSource:0}: Error finding container 2b9301feac7e7c08023a4f3a1ce2b786041992ffbaa1b6e547349092b91f29ef: Status 404 returned error can't find the container with id 2b9301feac7e7c08023a4f3a1ce2b786041992ffbaa1b6e547349092b91f29ef
Jul 15 05:10:37.057313 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.057222    2581 generic.go:356] "Generic (PLEG): container finished" podID="89cd258c-b6aa-4ec4-94ff-e0611a1ac278" containerID="66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794" exitCode=0
Jul 15 05:10:37.057313 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.057272    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"
Jul 15 05:10:37.057313 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.057297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv" event={"ID":"89cd258c-b6aa-4ec4-94ff-e0611a1ac278","Type":"ContainerDied","Data":"66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794"}
Jul 15 05:10:37.057781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.057346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv" event={"ID":"89cd258c-b6aa-4ec4-94ff-e0611a1ac278","Type":"ContainerDied","Data":"edbfeace802291c592dd35d879f02940d54068199170029f4824328cef221795"}
Jul 15 05:10:37.057781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.057373    2581 scope.go:117] "RemoveContainer" containerID="66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794"
Jul 15 05:10:37.059112 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.059085    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p" event={"ID":"00b909e7-1ddd-4078-8da2-3fccf0ab7209","Type":"ContainerStarted","Data":"622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c"}
Jul 15 05:10:37.059238 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.059131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p" event={"ID":"00b909e7-1ddd-4078-8da2-3fccf0ab7209","Type":"ContainerStarted","Data":"2b9301feac7e7c08023a4f3a1ce2b786041992ffbaa1b6e547349092b91f29ef"}
Jul 15 05:10:37.059238 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.059159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:37.066040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.066026    2581 scope.go:117] "RemoveContainer" containerID="66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794"
Jul 15 05:10:37.066290 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:37.066274    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794\": container with ID starting with 66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794 not found: ID does not exist" containerID="66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794"
Jul 15 05:10:37.066357 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.066298    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794"} err="failed to get container status \"66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794\": rpc error: code = NotFound desc = could not find container \"66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794\": container with ID starting with 66ee76926a5741dc703eddf5bf7c65cc741c019e99b78fd4050d67d96324d794 not found: ID does not exist"
Jul 15 05:10:37.098915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.098873    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p" podStartSLOduration=1.098859915 podStartE2EDuration="1.098859915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:37.096622033 +0000 UTC m=+715.264218705" watchObservedRunningTime="2026-07-15 05:10:37.098859915 +0000 UTC m=+715.266456587"
Jul 15 05:10:37.121702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.121674    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"]
Jul 15 05:10:37.143040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:37.143016    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j4lxv"]
Jul 15 05:10:38.388008 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:38.387974    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89cd258c-b6aa-4ec4-94ff-e0611a1ac278" path="/var/lib/kubelet/pods/89cd258c-b6aa-4ec4-94ff-e0611a1ac278/volumes"
Jul 15 05:10:40.632298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:40.632271    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:40.632677 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:40.632590    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:40.637140 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:40.637118    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:41.088444 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:41.088419    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9949b5ff-qnhzh"
Jul 15 05:10:41.134191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:41.134166    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65887b9d75-zk6s7"]
Jul 15 05:10:48.072674 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.072642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:10:48.125486 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.125460    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"]
Jul 15 05:10:48.125703 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.125675    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr" podUID="b40ce13d-5697-4be2-b146-a0cb1342df91" containerName="manager" containerID="cri-o://353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1" gracePeriod=10
Jul 15 05:10:48.368828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.368805    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:48.521130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.521098    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9bc8\" (UniqueName: \"kubernetes.io/projected/b40ce13d-5697-4be2-b146-a0cb1342df91-kube-api-access-s9bc8\") pod \"b40ce13d-5697-4be2-b146-a0cb1342df91\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") "
Jul 15 05:10:48.521304 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.521161    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b40ce13d-5697-4be2-b146-a0cb1342df91-extensions-socket-volume\") pod \"b40ce13d-5697-4be2-b146-a0cb1342df91\" (UID: \"b40ce13d-5697-4be2-b146-a0cb1342df91\") "
Jul 15 05:10:48.521615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.521574    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b40ce13d-5697-4be2-b146-a0cb1342df91-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b40ce13d-5697-4be2-b146-a0cb1342df91" (UID: "b40ce13d-5697-4be2-b146-a0cb1342df91"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:10:48.523166 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.523143    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40ce13d-5697-4be2-b146-a0cb1342df91-kube-api-access-s9bc8" (OuterVolumeSpecName: "kube-api-access-s9bc8") pod "b40ce13d-5697-4be2-b146-a0cb1342df91" (UID: "b40ce13d-5697-4be2-b146-a0cb1342df91"). InnerVolumeSpecName "kube-api-access-s9bc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:10:48.622641 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.622573    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9bc8\" (UniqueName: \"kubernetes.io/projected/b40ce13d-5697-4be2-b146-a0cb1342df91-kube-api-access-s9bc8\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:48.622641 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:48.622606    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b40ce13d-5697-4be2-b146-a0cb1342df91-extensions-socket-volume\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:10:49.123442 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.123408    2581 generic.go:356] "Generic (PLEG): container finished" podID="b40ce13d-5697-4be2-b146-a0cb1342df91" containerID="353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1" exitCode=0
Jul 15 05:10:49.123812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.123475    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"
Jul 15 05:10:49.123812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.123476    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr" event={"ID":"b40ce13d-5697-4be2-b146-a0cb1342df91","Type":"ContainerDied","Data":"353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1"}
Jul 15 05:10:49.123812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.123559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr" event={"ID":"b40ce13d-5697-4be2-b146-a0cb1342df91","Type":"ContainerDied","Data":"c3b2483f5d0a4120f815117d9f65363cbed1ec1d21a59821f589eac298f1310c"}
Jul 15 05:10:49.123812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.123580    2581 scope.go:117] "RemoveContainer" containerID="353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1"
Jul 15 05:10:49.132772 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.132580    2581 scope.go:117] "RemoveContainer" containerID="353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1"
Jul 15 05:10:49.132876 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:10:49.132857    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1\": container with ID starting with 353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1 not found: ID does not exist" containerID="353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1"
Jul 15 05:10:49.132918 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.132884    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1"} err="failed to get container status \"353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1\": rpc error: code = NotFound desc = could not find container \"353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1\": container with ID starting with 353c3733b90dc3d55fd21f94a0f314b1ddd6b8d878613570b41dacc864673ec1 not found: ID does not exist"
Jul 15 05:10:49.145412 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.145386    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"]
Jul 15 05:10:49.155016 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:49.154992    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-n5vxr"]
Jul 15 05:10:50.387702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:50.387667    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b40ce13d-5697-4be2-b146-a0cb1342df91" path="/var/lib/kubelet/pods/b40ce13d-5697-4be2-b146-a0cb1342df91/volumes"
Jul 15 05:10:52.391387 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.391356    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"]
Jul 15 05:10:52.391762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.391676    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cd258c-b6aa-4ec4-94ff-e0611a1ac278" containerName="manager"
Jul 15 05:10:52.391762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.391686    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cd258c-b6aa-4ec4-94ff-e0611a1ac278" containerName="manager"
Jul 15 05:10:52.391762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.391703    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b40ce13d-5697-4be2-b146-a0cb1342df91" containerName="manager"
Jul 15 05:10:52.391762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.391708    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b40ce13d-5697-4be2-b146-a0cb1342df91" containerName="manager"
Jul 15 05:10:52.391762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.391763    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cd258c-b6aa-4ec4-94ff-e0611a1ac278" containerName="manager"
Jul 15 05:10:52.391931 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.391773    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b40ce13d-5697-4be2-b146-a0cb1342df91" containerName="manager"
Jul 15 05:10:52.394660 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.394644    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.396970 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.396949    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-52mwv\""
Jul 15 05:10:52.406867 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.406848    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"]
Jul 15 05:10:52.551409 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551409 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551594 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrmmc\" (UniqueName: \"kubernetes.io/projected/2b2078b7-4b31-42c0-b440-ccfe7c196996-kube-api-access-zrmmc\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551594 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551594 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551594 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b2078b7-4b31-42c0-b440-ccfe7c196996-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551716 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.551793 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.551756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.652654 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652582    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.652654 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.652654 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrmmc\" (UniqueName: \"kubernetes.io/projected/2b2078b7-4b31-42c0-b440-ccfe7c196996-kube-api-access-zrmmc\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.652884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.652884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.652884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652726    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b2078b7-4b31-42c0-b440-ccfe7c196996-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.652884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.653083 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.653051    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.653154 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.652848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.653242 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.653160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.653242 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.653194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.653358 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.653236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.653442 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.653421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.653577 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.653555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b2078b7-4b31-42c0-b440-ccfe7c196996-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.655547 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.655522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.655650 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.655635    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.661155 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.661130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b2078b7-4b31-42c0-b440-ccfe7c196996-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.661410 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.661395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrmmc\" (UniqueName: \"kubernetes.io/projected/2b2078b7-4b31-42c0-b440-ccfe7c196996-kube-api-access-zrmmc\") pod \"maas-default-gateway-openshift-default-56654dcf7-6czc5\" (UID: \"2b2078b7-4b31-42c0-b440-ccfe7c196996\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.704373 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.704346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:52.829922 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.829898    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"]
Jul 15 05:10:52.831461 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:10:52.831432    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b2078b7_4b31_42c0_b440_ccfe7c196996.slice/crio-b500f731de8f9661fe3089bf32550e3493be47933340bb838ef0e3a25e14493f WatchSource:0}: Error finding container b500f731de8f9661fe3089bf32550e3493be47933340bb838ef0e3a25e14493f: Status 404 returned error can't find the container with id b500f731de8f9661fe3089bf32550e3493be47933340bb838ef0e3a25e14493f
Jul 15 05:10:52.833728 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.833688    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:10:52.833818 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.833770    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:10:52.833858 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:52.833813    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:10:53.152697 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:53.152659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" event={"ID":"2b2078b7-4b31-42c0-b440-ccfe7c196996","Type":"ContainerStarted","Data":"f97ae2e481842f64334a11724ac0691cb2bcf742777e5065af70cb893c9becdd"}
Jul 15 05:10:53.152855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:53.152751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" event={"ID":"2b2078b7-4b31-42c0-b440-ccfe7c196996","Type":"ContainerStarted","Data":"b500f731de8f9661fe3089bf32550e3493be47933340bb838ef0e3a25e14493f"}
Jul 15 05:10:53.175613 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:53.175561    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" podStartSLOduration=1.175548062 podStartE2EDuration="1.175548062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:53.174118554 +0000 UTC m=+731.341715226" watchObservedRunningTime="2026-07-15 05:10:53.175548062 +0000 UTC m=+731.343144734"
Jul 15 05:10:53.705376 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:53.705344    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:53.709538 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:53.709515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:54.156940 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:54.156911    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:10:54.157757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:10:54.157737    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:11:05.621384 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.621351    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"]
Jul 15 05:11:05.624886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.624863    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:05.627499 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.627478    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-26c86\""
Jul 15 05:11:05.627602 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.627484    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 05:11:05.632026 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.632006    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"]
Jul 15 05:11:05.658652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.658630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65e821e5-5e50-459c-8c24-9f47de4b688e-config-file\") pod \"limitador-limitador-64c94fd64f-qr6bh\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:05.658752 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.658699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjlb6\" (UniqueName: \"kubernetes.io/projected/65e821e5-5e50-459c-8c24-9f47de4b688e-kube-api-access-xjlb6\") pod \"limitador-limitador-64c94fd64f-qr6bh\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:05.723119 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.723081    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"]
Jul 15 05:11:05.759114 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.759084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65e821e5-5e50-459c-8c24-9f47de4b688e-config-file\") pod \"limitador-limitador-64c94fd64f-qr6bh\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:05.759250 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.759137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjlb6\" (UniqueName: \"kubernetes.io/projected/65e821e5-5e50-459c-8c24-9f47de4b688e-kube-api-access-xjlb6\") pod \"limitador-limitador-64c94fd64f-qr6bh\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:05.759760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.759739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65e821e5-5e50-459c-8c24-9f47de4b688e-config-file\") pod \"limitador-limitador-64c94fd64f-qr6bh\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:05.767014 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.766991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjlb6\" (UniqueName: \"kubernetes.io/projected/65e821e5-5e50-459c-8c24-9f47de4b688e-kube-api-access-xjlb6\") pod \"limitador-limitador-64c94fd64f-qr6bh\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:05.936036 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:05.935960    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:06.069874 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.069851    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"]
Jul 15 05:11:06.072073 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:11:06.072048    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e821e5_5e50_459c_8c24_9f47de4b688e.slice/crio-41b43e1dd9d030b82f677ae5bddc11e2a39f9e187c59cf697ea014d742b13887 WatchSource:0}: Error finding container 41b43e1dd9d030b82f677ae5bddc11e2a39f9e187c59cf697ea014d742b13887: Status 404 returned error can't find the container with id 41b43e1dd9d030b82f677ae5bddc11e2a39f9e187c59cf697ea014d742b13887
Jul 15 05:11:06.154667 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.154627    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65887b9d75-zk6s7" podUID="0dac9312-a420-45ef-8654-68ccdad3172d" containerName="console" containerID="cri-o://499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349" gracePeriod=15
Jul 15 05:11:06.214546 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.214481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh" event={"ID":"65e821e5-5e50-459c-8c24-9f47de4b688e","Type":"ContainerStarted","Data":"41b43e1dd9d030b82f677ae5bddc11e2a39f9e187c59cf697ea014d742b13887"}
Jul 15 05:11:06.390230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.390178    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:11:06.423409 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.423383    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-lz56x"]
Jul 15 05:11:06.423797 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.423781    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dac9312-a420-45ef-8654-68ccdad3172d" containerName="console"
Jul 15 05:11:06.423797 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.423797    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dac9312-a420-45ef-8654-68ccdad3172d" containerName="console"
Jul 15 05:11:06.424115 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.423907    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dac9312-a420-45ef-8654-68ccdad3172d" containerName="console"
Jul 15 05:11:06.426631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.426613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lz56x"
Jul 15 05:11:06.429282 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.429246    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2g29f\""
Jul 15 05:11:06.435651 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.435612    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-lz56x"]
Jul 15 05:11:06.465156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465099    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-trusted-ca-bundle\") pod \"0dac9312-a420-45ef-8654-68ccdad3172d\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") "
Jul 15 05:11:06.465156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465129    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-oauth-config\") pod \"0dac9312-a420-45ef-8654-68ccdad3172d\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") "
Jul 15 05:11:06.465328 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465162    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-service-ca\") pod \"0dac9312-a420-45ef-8654-68ccdad3172d\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") "
Jul 15 05:11:06.465328 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465184    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-console-config\") pod \"0dac9312-a420-45ef-8654-68ccdad3172d\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") "
Jul 15 05:11:06.465328 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465289    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-oauth-serving-cert\") pod \"0dac9312-a420-45ef-8654-68ccdad3172d\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") "
Jul 15 05:11:06.465487 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-serving-cert\") pod \"0dac9312-a420-45ef-8654-68ccdad3172d\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") "
Jul 15 05:11:06.465659 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465636    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-console-config" (OuterVolumeSpecName: "console-config") pod "0dac9312-a420-45ef-8654-68ccdad3172d" (UID: "0dac9312-a420-45ef-8654-68ccdad3172d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:11:06.465745 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfz98\" (UniqueName: \"kubernetes.io/projected/0dac9312-a420-45ef-8654-68ccdad3172d-kube-api-access-sfz98\") pod \"0dac9312-a420-45ef-8654-68ccdad3172d\" (UID: \"0dac9312-a420-45ef-8654-68ccdad3172d\") "
Jul 15 05:11:06.465745 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465708    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-service-ca" (OuterVolumeSpecName: "service-ca") pod "0dac9312-a420-45ef-8654-68ccdad3172d" (UID: "0dac9312-a420-45ef-8654-68ccdad3172d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:11:06.465745 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.465734    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0dac9312-a420-45ef-8654-68ccdad3172d" (UID: "0dac9312-a420-45ef-8654-68ccdad3172d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:11:06.466306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.466103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2stp\" (UniqueName: \"kubernetes.io/projected/1eeb50d2-0cda-4fff-8b93-586f9e1834c9-kube-api-access-h2stp\") pod \"authorino-58cd955675-lz56x\" (UID: \"1eeb50d2-0cda-4fff-8b93-586f9e1834c9\") " pod="kuadrant-system/authorino-58cd955675-lz56x"
Jul 15 05:11:06.466306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.466177    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-trusted-ca-bundle\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:06.466306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.466195    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:06.466306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.466244    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-console-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:06.466612 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.466594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0dac9312-a420-45ef-8654-68ccdad3172d" (UID: "0dac9312-a420-45ef-8654-68ccdad3172d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:11:06.467608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.467582    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0dac9312-a420-45ef-8654-68ccdad3172d" (UID: "0dac9312-a420-45ef-8654-68ccdad3172d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:11:06.467781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.467764    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0dac9312-a420-45ef-8654-68ccdad3172d" (UID: "0dac9312-a420-45ef-8654-68ccdad3172d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:11:06.467845 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.467775    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dac9312-a420-45ef-8654-68ccdad3172d-kube-api-access-sfz98" (OuterVolumeSpecName: "kube-api-access-sfz98") pod "0dac9312-a420-45ef-8654-68ccdad3172d" (UID: "0dac9312-a420-45ef-8654-68ccdad3172d"). InnerVolumeSpecName "kube-api-access-sfz98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:11:06.567080 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.567057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2stp\" (UniqueName: \"kubernetes.io/projected/1eeb50d2-0cda-4fff-8b93-586f9e1834c9-kube-api-access-h2stp\") pod \"authorino-58cd955675-lz56x\" (UID: \"1eeb50d2-0cda-4fff-8b93-586f9e1834c9\") " pod="kuadrant-system/authorino-58cd955675-lz56x"
Jul 15 05:11:06.567181 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.567116    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfz98\" (UniqueName: \"kubernetes.io/projected/0dac9312-a420-45ef-8654-68ccdad3172d-kube-api-access-sfz98\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:06.567181 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.567133    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-oauth-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:06.567181 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.567146    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dac9312-a420-45ef-8654-68ccdad3172d-oauth-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:06.567181 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.567158    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dac9312-a420-45ef-8654-68ccdad3172d-console-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:06.574801 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.574777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2stp\" (UniqueName: \"kubernetes.io/projected/1eeb50d2-0cda-4fff-8b93-586f9e1834c9-kube-api-access-h2stp\") pod \"authorino-58cd955675-lz56x\" (UID: \"1eeb50d2-0cda-4fff-8b93-586f9e1834c9\") " pod="kuadrant-system/authorino-58cd955675-lz56x"
Jul 15 05:11:06.736467 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.736392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lz56x"
Jul 15 05:11:06.897167 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:11:06.897134    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1eeb50d2_0cda_4fff_8b93_586f9e1834c9.slice/crio-b6621faa717d410d1aef31fb2804d119887db57776bc83de5953788535e0f75e WatchSource:0}: Error finding container b6621faa717d410d1aef31fb2804d119887db57776bc83de5953788535e0f75e: Status 404 returned error can't find the container with id b6621faa717d410d1aef31fb2804d119887db57776bc83de5953788535e0f75e
Jul 15 05:11:06.903081 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:06.903029    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-lz56x"]
Jul 15 05:11:07.221940 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.221879    2581 generic.go:356] "Generic (PLEG): container finished" podID="0dac9312-a420-45ef-8654-68ccdad3172d" containerID="499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349" exitCode=0
Jul 15 05:11:07.221940 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.221923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65887b9d75-zk6s7" event={"ID":"0dac9312-a420-45ef-8654-68ccdad3172d","Type":"ContainerDied","Data":"499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349"}
Jul 15 05:11:07.222928 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.221939    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65887b9d75-zk6s7"
Jul 15 05:11:07.222928 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.221982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65887b9d75-zk6s7" event={"ID":"0dac9312-a420-45ef-8654-68ccdad3172d","Type":"ContainerDied","Data":"20dac2ad03ef7c34d98bbefce39dfe85eb5f986b505e9a102ab4c3169938a271"}
Jul 15 05:11:07.222928 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.222016    2581 scope.go:117] "RemoveContainer" containerID="499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349"
Jul 15 05:11:07.223738 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.223681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lz56x" event={"ID":"1eeb50d2-0cda-4fff-8b93-586f9e1834c9","Type":"ContainerStarted","Data":"b6621faa717d410d1aef31fb2804d119887db57776bc83de5953788535e0f75e"}
Jul 15 05:11:07.233103 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.232903    2581 scope.go:117] "RemoveContainer" containerID="499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349"
Jul 15 05:11:07.233780 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:11:07.233752    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349\": container with ID starting with 499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349 not found: ID does not exist" containerID="499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349"
Jul 15 05:11:07.233900 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.233789    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349"} err="failed to get container status \"499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349\": rpc error: code = NotFound desc = could not find container \"499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349\": container with ID starting with 499329cfe9ab7b9cf9fdceaefc706067f02584c27d25f029efc01bdf34269349 not found: ID does not exist"
Jul 15 05:11:07.248981 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.248961    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65887b9d75-zk6s7"]
Jul 15 05:11:07.252672 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:07.252651    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65887b9d75-zk6s7"]
Jul 15 05:11:08.389452 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:08.389419    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dac9312-a420-45ef-8654-68ccdad3172d" path="/var/lib/kubelet/pods/0dac9312-a420-45ef-8654-68ccdad3172d/volumes"
Jul 15 05:11:10.642056 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:10.642032    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-lz56x"]
Jul 15 05:11:11.246565 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.246522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh" event={"ID":"65e821e5-5e50-459c-8c24-9f47de4b688e","Type":"ContainerStarted","Data":"34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521"}
Jul 15 05:11:11.246749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.246676    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:11.247941 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.247920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lz56x" event={"ID":"1eeb50d2-0cda-4fff-8b93-586f9e1834c9","Type":"ContainerStarted","Data":"d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55"}
Jul 15 05:11:11.248057 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.248019    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-lz56x" podUID="1eeb50d2-0cda-4fff-8b93-586f9e1834c9" containerName="authorino" containerID="cri-o://d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55" gracePeriod=30
Jul 15 05:11:11.265913 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.265873    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh" podStartSLOduration=1.749828634 podStartE2EDuration="6.265858784s" totalImagesPullingTime="4.51603015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:11:11.264653816 +0000 UTC m=+749.432250489" watchObservedRunningTime="2026-07-15 05:11:11.265858784 +0000 UTC m=+749.433455482"
Jul 15 05:11:11.279075 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.279032    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-lz56x" podStartSLOduration=1.581938858 podStartE2EDuration="5.279015723s" totalImagesPullingTime="3.697076865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:11:11.278732865 +0000 UTC m=+749.446329538" watchObservedRunningTime="2026-07-15 05:11:11.279015723 +0000 UTC m=+749.446612396"
Jul 15 05:11:11.491373 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.491350    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lz56x"
Jul 15 05:11:11.618553 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.618524    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2stp\" (UniqueName: \"kubernetes.io/projected/1eeb50d2-0cda-4fff-8b93-586f9e1834c9-kube-api-access-h2stp\") pod \"1eeb50d2-0cda-4fff-8b93-586f9e1834c9\" (UID: \"1eeb50d2-0cda-4fff-8b93-586f9e1834c9\") "
Jul 15 05:11:11.620538 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.620514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eeb50d2-0cda-4fff-8b93-586f9e1834c9-kube-api-access-h2stp" (OuterVolumeSpecName: "kube-api-access-h2stp") pod "1eeb50d2-0cda-4fff-8b93-586f9e1834c9" (UID: "1eeb50d2-0cda-4fff-8b93-586f9e1834c9"). InnerVolumeSpecName "kube-api-access-h2stp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:11:11.719159 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:11.719134    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2stp\" (UniqueName: \"kubernetes.io/projected/1eeb50d2-0cda-4fff-8b93-586f9e1834c9-kube-api-access-h2stp\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:12.255985 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.255948    2581 generic.go:356] "Generic (PLEG): container finished" podID="1eeb50d2-0cda-4fff-8b93-586f9e1834c9" containerID="d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55" exitCode=0
Jul 15 05:11:12.256156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.255990    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lz56x"
Jul 15 05:11:12.256156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.256035    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lz56x" event={"ID":"1eeb50d2-0cda-4fff-8b93-586f9e1834c9","Type":"ContainerDied","Data":"d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55"}
Jul 15 05:11:12.256156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.256080    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lz56x" event={"ID":"1eeb50d2-0cda-4fff-8b93-586f9e1834c9","Type":"ContainerDied","Data":"b6621faa717d410d1aef31fb2804d119887db57776bc83de5953788535e0f75e"}
Jul 15 05:11:12.256156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.256101    2581 scope.go:117] "RemoveContainer" containerID="d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55"
Jul 15 05:11:12.264831 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.264816    2581 scope.go:117] "RemoveContainer" containerID="d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55"
Jul 15 05:11:12.265089 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:11:12.265071    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55\": container with ID starting with d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55 not found: ID does not exist" containerID="d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55"
Jul 15 05:11:12.265141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.265096    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55"} err="failed to get container status \"d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55\": rpc error: code = NotFound desc = could not find container \"d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55\": container with ID starting with d482a3888edd02f7f12f22c05d10dc5eb355735795e2e02f8613a046628f3d55 not found: ID does not exist"
Jul 15 05:11:12.276936 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.276913    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-lz56x"]
Jul 15 05:11:12.280520 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.280501    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-lz56x"]
Jul 15 05:11:12.387638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:12.387602    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eeb50d2-0cda-4fff-8b93-586f9e1834c9" path="/var/lib/kubelet/pods/1eeb50d2-0cda-4fff-8b93-586f9e1834c9/volumes"
Jul 15 05:11:21.120256 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:21.120220    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"]
Jul 15 05:11:21.120650 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:21.120444    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh" podUID="65e821e5-5e50-459c-8c24-9f47de4b688e" containerName="limitador" containerID="cri-o://34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521" gracePeriod=30
Jul 15 05:11:21.121312 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:21.121271    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:22.062796 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.062769    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:22.197295 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.197194    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjlb6\" (UniqueName: \"kubernetes.io/projected/65e821e5-5e50-459c-8c24-9f47de4b688e-kube-api-access-xjlb6\") pod \"65e821e5-5e50-459c-8c24-9f47de4b688e\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") "
Jul 15 05:11:22.197295 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.197287    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65e821e5-5e50-459c-8c24-9f47de4b688e-config-file\") pod \"65e821e5-5e50-459c-8c24-9f47de4b688e\" (UID: \"65e821e5-5e50-459c-8c24-9f47de4b688e\") "
Jul 15 05:11:22.197653 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.197597    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65e821e5-5e50-459c-8c24-9f47de4b688e-config-file" (OuterVolumeSpecName: "config-file") pod "65e821e5-5e50-459c-8c24-9f47de4b688e" (UID: "65e821e5-5e50-459c-8c24-9f47de4b688e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:11:22.199140 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.199109    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65e821e5-5e50-459c-8c24-9f47de4b688e-kube-api-access-xjlb6" (OuterVolumeSpecName: "kube-api-access-xjlb6") pod "65e821e5-5e50-459c-8c24-9f47de4b688e" (UID: "65e821e5-5e50-459c-8c24-9f47de4b688e"). InnerVolumeSpecName "kube-api-access-xjlb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:11:22.298730 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.298699    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjlb6\" (UniqueName: \"kubernetes.io/projected/65e821e5-5e50-459c-8c24-9f47de4b688e-kube-api-access-xjlb6\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:22.298730 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.298725    2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65e821e5-5e50-459c-8c24-9f47de4b688e-config-file\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:22.306006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.305964    2581 generic.go:356] "Generic (PLEG): container finished" podID="65e821e5-5e50-459c-8c24-9f47de4b688e" containerID="34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521" exitCode=0
Jul 15 05:11:22.306127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.306017    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"
Jul 15 05:11:22.306127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.306059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh" event={"ID":"65e821e5-5e50-459c-8c24-9f47de4b688e","Type":"ContainerDied","Data":"34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521"}
Jul 15 05:11:22.306127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.306116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qr6bh" event={"ID":"65e821e5-5e50-459c-8c24-9f47de4b688e","Type":"ContainerDied","Data":"41b43e1dd9d030b82f677ae5bddc11e2a39f9e187c59cf697ea014d742b13887"}
Jul 15 05:11:22.306239 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.306142    2581 scope.go:117] "RemoveContainer" containerID="34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521"
Jul 15 05:11:22.315753 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.315737    2581 scope.go:117] "RemoveContainer" containerID="34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521"
Jul 15 05:11:22.315992 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:11:22.315972    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521\": container with ID starting with 34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521 not found: ID does not exist" containerID="34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521"
Jul 15 05:11:22.316038 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.316000    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521"} err="failed to get container status \"34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521\": rpc error: code = NotFound desc = could not find container \"34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521\": container with ID starting with 34294e66a27055017f828b5b3360857f45bdd82e72d22385c1f9ab8f6cbb7521 not found: ID does not exist"
Jul 15 05:11:22.327758 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.327736    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"]
Jul 15 05:11:22.332162 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.332142    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qr6bh"]
Jul 15 05:11:22.387927 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:22.387904    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65e821e5-5e50-459c-8c24-9f47de4b688e" path="/var/lib/kubelet/pods/65e821e5-5e50-459c-8c24-9f47de4b688e/volumes"
Jul 15 05:11:26.054491 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.054464    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"]
Jul 15 05:11:26.054833 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.054815    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eeb50d2-0cda-4fff-8b93-586f9e1834c9" containerName="authorino"
Jul 15 05:11:26.054833 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.054829    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eeb50d2-0cda-4fff-8b93-586f9e1834c9" containerName="authorino"
Jul 15 05:11:26.054937 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.054849    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65e821e5-5e50-459c-8c24-9f47de4b688e" containerName="limitador"
Jul 15 05:11:26.054937 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.054855    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="65e821e5-5e50-459c-8c24-9f47de4b688e" containerName="limitador"
Jul 15 05:11:26.054937 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.054919    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="65e821e5-5e50-459c-8c24-9f47de4b688e" containerName="limitador"
Jul 15 05:11:26.054937 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.054929    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eeb50d2-0cda-4fff-8b93-586f9e1834c9" containerName="authorino"
Jul 15 05:11:26.060655 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.060634    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.064085 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.063841    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 05:11:26.064085 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.063868    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 05:11:26.064085 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.063847    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zf27f\""
Jul 15 05:11:26.064085 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.064041    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 05:11:26.064703 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.064683    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"]
Jul 15 05:11:26.125500 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.125474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0d669fb4-5dc1-4f50-81ea-caaa168d9fca-data\") pod \"postgres-69fb4594cf-v99mt\" (UID: \"0d669fb4-5dc1-4f50-81ea-caaa168d9fca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.125596 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.125506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg5js\" (UniqueName: \"kubernetes.io/projected/0d669fb4-5dc1-4f50-81ea-caaa168d9fca-kube-api-access-xg5js\") pod \"postgres-69fb4594cf-v99mt\" (UID: \"0d669fb4-5dc1-4f50-81ea-caaa168d9fca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.226254 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.226227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0d669fb4-5dc1-4f50-81ea-caaa168d9fca-data\") pod \"postgres-69fb4594cf-v99mt\" (UID: \"0d669fb4-5dc1-4f50-81ea-caaa168d9fca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.226347 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.226259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5js\" (UniqueName: \"kubernetes.io/projected/0d669fb4-5dc1-4f50-81ea-caaa168d9fca-kube-api-access-xg5js\") pod \"postgres-69fb4594cf-v99mt\" (UID: \"0d669fb4-5dc1-4f50-81ea-caaa168d9fca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.226631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.226607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0d669fb4-5dc1-4f50-81ea-caaa168d9fca-data\") pod \"postgres-69fb4594cf-v99mt\" (UID: \"0d669fb4-5dc1-4f50-81ea-caaa168d9fca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.234288 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.234266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5js\" (UniqueName: \"kubernetes.io/projected/0d669fb4-5dc1-4f50-81ea-caaa168d9fca-kube-api-access-xg5js\") pod \"postgres-69fb4594cf-v99mt\" (UID: \"0d669fb4-5dc1-4f50-81ea-caaa168d9fca\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.372666 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.372643    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:26.501412 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:26.501384    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"]
Jul 15 05:11:26.503466 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:11:26.503441    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d669fb4_5dc1_4f50_81ea_caaa168d9fca.slice/crio-440a429bc7ea4b1eccb4276bbf15a69ed777af0a000e567ca91706b727c04a1e WatchSource:0}: Error finding container 440a429bc7ea4b1eccb4276bbf15a69ed777af0a000e567ca91706b727c04a1e: Status 404 returned error can't find the container with id 440a429bc7ea4b1eccb4276bbf15a69ed777af0a000e567ca91706b727c04a1e
Jul 15 05:11:27.335252 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:27.335215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt" event={"ID":"0d669fb4-5dc1-4f50-81ea-caaa168d9fca","Type":"ContainerStarted","Data":"440a429bc7ea4b1eccb4276bbf15a69ed777af0a000e567ca91706b727c04a1e"}
Jul 15 05:11:32.360961 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:32.360927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt" event={"ID":"0d669fb4-5dc1-4f50-81ea-caaa168d9fca","Type":"ContainerStarted","Data":"59ce646bd9f599be2c6a38d6e84b65937dbfe05a6d46dd51265da22008499f05"}
Jul 15 05:11:32.361394 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:32.361067    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:32.379877 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:32.379837    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt" podStartSLOduration=1.22951591 podStartE2EDuration="6.379824678s" totalImagesPullingTime="5.150308768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:11:32.379783518 +0000 UTC m=+770.547380192" watchObservedRunningTime="2026-07-15 05:11:32.379824678 +0000 UTC m=+770.547421352"
Jul 15 05:11:38.395828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:38.395801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-v99mt"
Jul 15 05:11:38.930903 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:38.930872    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj4hm"]
Jul 15 05:11:38.933422 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:38.933402    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj4hm"
Jul 15 05:11:38.935757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:38.935738    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2g29f\""
Jul 15 05:11:38.939825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:38.939801    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj4hm"]
Jul 15 05:11:39.035115 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.035086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5kvv\" (UniqueName: \"kubernetes.io/projected/91b1311c-c25c-4a71-8ff3-a7a7941f3a76-kube-api-access-r5kvv\") pod \"authorino-5d4b67c446-kj4hm\" (UID: \"91b1311c-c25c-4a71-8ff3-a7a7941f3a76\") " pod="kuadrant-system/authorino-5d4b67c446-kj4hm"
Jul 15 05:11:39.113924 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.113898    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj4hm"]
Jul 15 05:11:39.114132 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:11:39.114115    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-r5kvv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-kj4hm" podUID="91b1311c-c25c-4a71-8ff3-a7a7941f3a76"
Jul 15 05:11:39.136468 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.136439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5kvv\" (UniqueName: \"kubernetes.io/projected/91b1311c-c25c-4a71-8ff3-a7a7941f3a76-kube-api-access-r5kvv\") pod \"authorino-5d4b67c446-kj4hm\" (UID: \"91b1311c-c25c-4a71-8ff3-a7a7941f3a76\") " pod="kuadrant-system/authorino-5d4b67c446-kj4hm"
Jul 15 05:11:39.140761 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.140739    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-kk2r7"]
Jul 15 05:11:39.143514 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.143494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.144530 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.144509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5kvv\" (UniqueName: \"kubernetes.io/projected/91b1311c-c25c-4a71-8ff3-a7a7941f3a76-kube-api-access-r5kvv\") pod \"authorino-5d4b67c446-kj4hm\" (UID: \"91b1311c-c25c-4a71-8ff3-a7a7941f3a76\") " pod="kuadrant-system/authorino-5d4b67c446-kj4hm"
Jul 15 05:11:39.145827 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.145811    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 15 05:11:39.152378 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.152349    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-kk2r7"]
Jul 15 05:11:39.205144 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.205091    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-kk2r7"]
Jul 15 05:11:39.205330 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:11:39.205310    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-85qpd tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-kk2r7" podUID="3cffa885-0457-41e1-bfda-236f89d97151"
Jul 15 05:11:39.229690 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.229670    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-595bcff787-t48cd"]
Jul 15 05:11:39.232189 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.232173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.237185 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.237164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cffa885-0457-41e1-bfda-236f89d97151-tls-cert\") pod \"authorino-76877994-kk2r7\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") " pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.237298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.237217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85qpd\" (UniqueName: \"kubernetes.io/projected/3cffa885-0457-41e1-bfda-236f89d97151-kube-api-access-85qpd\") pod \"authorino-76877994-kk2r7\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") " pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.240423 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.240399    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595bcff787-t48cd"]
Jul 15 05:11:39.338564 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.338532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85qpd\" (UniqueName: \"kubernetes.io/projected/3cffa885-0457-41e1-bfda-236f89d97151-kube-api-access-85qpd\") pod \"authorino-76877994-kk2r7\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") " pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.338690 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.338579    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6fae417-ed08-407d-91af-bfcabee47f33-tls-cert\") pod \"authorino-595bcff787-t48cd\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") " pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.338690 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.338650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmxbm\" (UniqueName: \"kubernetes.io/projected/e6fae417-ed08-407d-91af-bfcabee47f33-kube-api-access-gmxbm\") pod \"authorino-595bcff787-t48cd\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") " pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.338690 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.338680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cffa885-0457-41e1-bfda-236f89d97151-tls-cert\") pod \"authorino-76877994-kk2r7\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") " pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.340974 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.340956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cffa885-0457-41e1-bfda-236f89d97151-tls-cert\") pod \"authorino-76877994-kk2r7\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") " pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.346400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.346382    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85qpd\" (UniqueName: \"kubernetes.io/projected/3cffa885-0457-41e1-bfda-236f89d97151-kube-api-access-85qpd\") pod \"authorino-76877994-kk2r7\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") " pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.400063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.400036    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj4hm"
Jul 15 05:11:39.400372 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.400036    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.406332 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.406313    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj4hm"
Jul 15 05:11:39.409367 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.409350    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:39.440038 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.440017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmxbm\" (UniqueName: \"kubernetes.io/projected/e6fae417-ed08-407d-91af-bfcabee47f33-kube-api-access-gmxbm\") pod \"authorino-595bcff787-t48cd\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") " pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.440162 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.440060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6fae417-ed08-407d-91af-bfcabee47f33-tls-cert\") pod \"authorino-595bcff787-t48cd\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") " pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.442432 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.442413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6fae417-ed08-407d-91af-bfcabee47f33-tls-cert\") pod \"authorino-595bcff787-t48cd\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") " pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.447911 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.447881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmxbm\" (UniqueName: \"kubernetes.io/projected/e6fae417-ed08-407d-91af-bfcabee47f33-kube-api-access-gmxbm\") pod \"authorino-595bcff787-t48cd\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") " pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.540611 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.540555    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cffa885-0457-41e1-bfda-236f89d97151-tls-cert\") pod \"3cffa885-0457-41e1-bfda-236f89d97151\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") "
Jul 15 05:11:39.540705 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.540622    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5kvv\" (UniqueName: \"kubernetes.io/projected/91b1311c-c25c-4a71-8ff3-a7a7941f3a76-kube-api-access-r5kvv\") pod \"91b1311c-c25c-4a71-8ff3-a7a7941f3a76\" (UID: \"91b1311c-c25c-4a71-8ff3-a7a7941f3a76\") "
Jul 15 05:11:39.540705 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.540691    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85qpd\" (UniqueName: \"kubernetes.io/projected/3cffa885-0457-41e1-bfda-236f89d97151-kube-api-access-85qpd\") pod \"3cffa885-0457-41e1-bfda-236f89d97151\" (UID: \"3cffa885-0457-41e1-bfda-236f89d97151\") "
Jul 15 05:11:39.540966 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.540941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:11:39.542674 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.542645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cffa885-0457-41e1-bfda-236f89d97151-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3cffa885-0457-41e1-bfda-236f89d97151" (UID: "3cffa885-0457-41e1-bfda-236f89d97151"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:11:39.542767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.542696    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cffa885-0457-41e1-bfda-236f89d97151-kube-api-access-85qpd" (OuterVolumeSpecName: "kube-api-access-85qpd") pod "3cffa885-0457-41e1-bfda-236f89d97151" (UID: "3cffa885-0457-41e1-bfda-236f89d97151"). InnerVolumeSpecName "kube-api-access-85qpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:11:39.542767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.542702    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b1311c-c25c-4a71-8ff3-a7a7941f3a76-kube-api-access-r5kvv" (OuterVolumeSpecName: "kube-api-access-r5kvv") pod "91b1311c-c25c-4a71-8ff3-a7a7941f3a76" (UID: "91b1311c-c25c-4a71-8ff3-a7a7941f3a76"). InnerVolumeSpecName "kube-api-access-r5kvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:11:39.641368 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.641339    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cffa885-0457-41e1-bfda-236f89d97151-tls-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:39.641368 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.641369    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5kvv\" (UniqueName: \"kubernetes.io/projected/91b1311c-c25c-4a71-8ff3-a7a7941f3a76-kube-api-access-r5kvv\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:39.641570 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.641383    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85qpd\" (UniqueName: \"kubernetes.io/projected/3cffa885-0457-41e1-bfda-236f89d97151-kube-api-access-85qpd\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:39.668645 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:39.668607    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595bcff787-t48cd"]
Jul 15 05:11:39.675633 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:11:39.675609    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6fae417_ed08_407d_91af_bfcabee47f33.slice/crio-4507c8f3c971aa512cd208a0a911cd9c3c62e80e7f35f7be84404f63a003c31c WatchSource:0}: Error finding container 4507c8f3c971aa512cd208a0a911cd9c3c62e80e7f35f7be84404f63a003c31c: Status 404 returned error can't find the container with id 4507c8f3c971aa512cd208a0a911cd9c3c62e80e7f35f7be84404f63a003c31c
Jul 15 05:11:40.405443 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.405412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595bcff787-t48cd" event={"ID":"e6fae417-ed08-407d-91af-bfcabee47f33","Type":"ContainerStarted","Data":"773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93"}
Jul 15 05:11:40.405735 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.405455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595bcff787-t48cd" event={"ID":"e6fae417-ed08-407d-91af-bfcabee47f33","Type":"ContainerStarted","Data":"4507c8f3c971aa512cd208a0a911cd9c3c62e80e7f35f7be84404f63a003c31c"}
Jul 15 05:11:40.405735 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.405554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kk2r7"
Jul 15 05:11:40.405735 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.405640    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj4hm"
Jul 15 05:11:40.426808 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.426766    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-595bcff787-t48cd" podStartSLOduration=1.084059753 podStartE2EDuration="1.426750085s" totalImagesPullingTime="342.690332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:11:40.424823323 +0000 UTC m=+778.592419994" watchObservedRunningTime="2026-07-15 05:11:40.426750085 +0000 UTC m=+778.594346788"
Jul 15 05:11:40.455234 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.455196    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj4hm"]
Jul 15 05:11:40.463223 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.463187    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj4hm"]
Jul 15 05:11:40.500942 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.500919    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-kk2r7"]
Jul 15 05:11:40.505934 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:40.505911    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-kk2r7"]
Jul 15 05:11:42.387896 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:42.387870    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cffa885-0457-41e1-bfda-236f89d97151" path="/var/lib/kubelet/pods/3cffa885-0457-41e1-bfda-236f89d97151/volumes"
Jul 15 05:11:42.388170 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:42.388075    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91b1311c-c25c-4a71-8ff3-a7a7941f3a76" path="/var/lib/kubelet/pods/91b1311c-c25c-4a71-8ff3-a7a7941f3a76/volumes"
Jul 15 05:11:45.189488 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.189451    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-698f8fdbf6-whlhg"]
Jul 15 05:11:45.193334 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.193310    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:45.195988 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.195961    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-94hdt\""
Jul 15 05:11:45.195988 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.195987    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 05:11:45.196136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.196006    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 05:11:45.200805 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.200783    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-698f8fdbf6-whlhg"]
Jul 15 05:11:45.294355 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.294328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvhc8\" (UniqueName: \"kubernetes.io/projected/253c642c-0f56-4f7c-a2aa-66649df09f64-kube-api-access-fvhc8\") pod \"maas-controller-698f8fdbf6-whlhg\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") " pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:45.294512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.294385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert\") pod \"maas-controller-698f8fdbf6-whlhg\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") " pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:45.367059 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.367034    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77f5789774-b5wh6"]
Jul 15 05:11:45.369821 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.369802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.380703 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.380681    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f5789774-b5wh6"]
Jul 15 05:11:45.394811 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.394788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvhc8\" (UniqueName: \"kubernetes.io/projected/253c642c-0f56-4f7c-a2aa-66649df09f64-kube-api-access-fvhc8\") pod \"maas-controller-698f8fdbf6-whlhg\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") " pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:45.394917 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.394845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert\") pod \"maas-controller-698f8fdbf6-whlhg\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") " pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:45.395002 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:11:45.394985    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 15 05:11:45.395065 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:11:45.395056    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert podName:253c642c-0f56-4f7c-a2aa-66649df09f64 nodeName:}" failed. No retries permitted until 2026-07-15 05:11:45.895039031 +0000 UTC m=+784.062635703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert") pod "maas-controller-698f8fdbf6-whlhg" (UID: "253c642c-0f56-4f7c-a2aa-66649df09f64") : secret "maas-controller-webhook-cert" not found
Jul 15 05:11:45.403673 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.403650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvhc8\" (UniqueName: \"kubernetes.io/projected/253c642c-0f56-4f7c-a2aa-66649df09f64-kube-api-access-fvhc8\") pod \"maas-controller-698f8fdbf6-whlhg\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") " pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:45.496178 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.496106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-webhook-cert\") pod \"maas-controller-77f5789774-b5wh6\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") " pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.496178 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.496168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cls4\" (UniqueName: \"kubernetes.io/projected/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-kube-api-access-5cls4\") pod \"maas-controller-77f5789774-b5wh6\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") " pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.597012 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.596993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-webhook-cert\") pod \"maas-controller-77f5789774-b5wh6\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") " pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.597110 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.597036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cls4\" (UniqueName: \"kubernetes.io/projected/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-kube-api-access-5cls4\") pod \"maas-controller-77f5789774-b5wh6\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") " pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.599298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.599274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-webhook-cert\") pod \"maas-controller-77f5789774-b5wh6\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") " pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.605459 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.605440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cls4\" (UniqueName: \"kubernetes.io/projected/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-kube-api-access-5cls4\") pod \"maas-controller-77f5789774-b5wh6\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") " pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.679413 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.679390    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:45.809142 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.809120    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f5789774-b5wh6"]
Jul 15 05:11:45.810439 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:11:45.810417    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2686e9_5aae_44c9_8ec7_5f94f74bbd92.slice/crio-df15139698067be9c67ea510ceac53ef56646efdf80de8233e6790feb22b74d9 WatchSource:0}: Error finding container df15139698067be9c67ea510ceac53ef56646efdf80de8233e6790feb22b74d9: Status 404 returned error can't find the container with id df15139698067be9c67ea510ceac53ef56646efdf80de8233e6790feb22b74d9
Jul 15 05:11:45.899808 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.899782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert\") pod \"maas-controller-698f8fdbf6-whlhg\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") " pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:45.902049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:45.902026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert\") pod \"maas-controller-698f8fdbf6-whlhg\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") " pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:46.104665 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:46.104636    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:46.239947 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:46.239918    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-698f8fdbf6-whlhg"]
Jul 15 05:11:46.241088 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:11:46.241061    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253c642c_0f56_4f7c_a2aa_66649df09f64.slice/crio-6f0309caa55e3528d238c5a14e555558a3b1a85456f3326bc8e1b4252c756345 WatchSource:0}: Error finding container 6f0309caa55e3528d238c5a14e555558a3b1a85456f3326bc8e1b4252c756345: Status 404 returned error can't find the container with id 6f0309caa55e3528d238c5a14e555558a3b1a85456f3326bc8e1b4252c756345
Jul 15 05:11:46.435772 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:46.435654    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f5789774-b5wh6" event={"ID":"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92","Type":"ContainerStarted","Data":"df15139698067be9c67ea510ceac53ef56646efdf80de8233e6790feb22b74d9"}
Jul 15 05:11:46.437315 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:46.437285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698f8fdbf6-whlhg" event={"ID":"253c642c-0f56-4f7c-a2aa-66649df09f64","Type":"ContainerStarted","Data":"6f0309caa55e3528d238c5a14e555558a3b1a85456f3326bc8e1b4252c756345"}
Jul 15 05:11:48.449478 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:48.449399    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f5789774-b5wh6" event={"ID":"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92","Type":"ContainerStarted","Data":"78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322"}
Jul 15 05:11:48.449950 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:48.449507    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:48.451007 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:48.450982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698f8fdbf6-whlhg" event={"ID":"253c642c-0f56-4f7c-a2aa-66649df09f64","Type":"ContainerStarted","Data":"d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d"}
Jul 15 05:11:48.451119 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:48.451041    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:48.468685 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:48.468641    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77f5789774-b5wh6" podStartSLOduration=1.175045594 podStartE2EDuration="3.468616858s" totalImagesPullingTime="2.293571264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:11:48.468050025 +0000 UTC m=+786.635646697" watchObservedRunningTime="2026-07-15 05:11:48.468616858 +0000 UTC m=+786.636213532"
Jul 15 05:11:48.485931 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:48.485885    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-698f8fdbf6-whlhg" podStartSLOduration=1.632476314 podStartE2EDuration="3.485870125s" totalImagesPullingTime="1.853393811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:11:48.485274406 +0000 UTC m=+786.652871081" watchObservedRunningTime="2026-07-15 05:11:48.485870125 +0000 UTC m=+786.653466817"
Jul 15 05:11:59.457381 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.457339    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:59.457773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.457560    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:11:59.515651 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.515621    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-698f8fdbf6-whlhg"]
Jul 15 05:11:59.516079 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.515850    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-698f8fdbf6-whlhg" podUID="253c642c-0f56-4f7c-a2aa-66649df09f64" containerName="manager" containerID="cri-o://d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d" gracePeriod=30
Jul 15 05:11:59.747683 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.747657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:11:59.806287 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.806261    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvhc8\" (UniqueName: \"kubernetes.io/projected/253c642c-0f56-4f7c-a2aa-66649df09f64-kube-api-access-fvhc8\") pod \"253c642c-0f56-4f7c-a2aa-66649df09f64\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") "
Jul 15 05:11:59.806382 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.806309    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert\") pod \"253c642c-0f56-4f7c-a2aa-66649df09f64\" (UID: \"253c642c-0f56-4f7c-a2aa-66649df09f64\") "
Jul 15 05:11:59.808326 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.808295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/253c642c-0f56-4f7c-a2aa-66649df09f64-kube-api-access-fvhc8" (OuterVolumeSpecName: "kube-api-access-fvhc8") pod "253c642c-0f56-4f7c-a2aa-66649df09f64" (UID: "253c642c-0f56-4f7c-a2aa-66649df09f64"). InnerVolumeSpecName "kube-api-access-fvhc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:11:59.808326 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.808300    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "253c642c-0f56-4f7c-a2aa-66649df09f64" (UID: "253c642c-0f56-4f7c-a2aa-66649df09f64"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:11:59.906992 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.906960    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvhc8\" (UniqueName: \"kubernetes.io/projected/253c642c-0f56-4f7c-a2aa-66649df09f64-kube-api-access-fvhc8\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:11:59.906992 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:11:59.906986    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253c642c-0f56-4f7c-a2aa-66649df09f64-webhook-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:12:00.507846 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.507807    2581 generic.go:356] "Generic (PLEG): container finished" podID="253c642c-0f56-4f7c-a2aa-66649df09f64" containerID="d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d" exitCode=0
Jul 15 05:12:00.508191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.507862    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-698f8fdbf6-whlhg"
Jul 15 05:12:00.508191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.507892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698f8fdbf6-whlhg" event={"ID":"253c642c-0f56-4f7c-a2aa-66649df09f64","Type":"ContainerDied","Data":"d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d"}
Jul 15 05:12:00.508191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.507965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-698f8fdbf6-whlhg" event={"ID":"253c642c-0f56-4f7c-a2aa-66649df09f64","Type":"ContainerDied","Data":"6f0309caa55e3528d238c5a14e555558a3b1a85456f3326bc8e1b4252c756345"}
Jul 15 05:12:00.508191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.507987    2581 scope.go:117] "RemoveContainer" containerID="d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d"
Jul 15 05:12:00.516132 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.516115    2581 scope.go:117] "RemoveContainer" containerID="d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d"
Jul 15 05:12:00.516411 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:12:00.516375    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d\": container with ID starting with d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d not found: ID does not exist" containerID="d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d"
Jul 15 05:12:00.516476 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.516404    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d"} err="failed to get container status \"d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d\": rpc error: code = NotFound desc = could not find container \"d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d\": container with ID starting with d2884ddf734c1d2020c5c60a8562e47567764881f9a24d78d302bfd7920e105d not found: ID does not exist"
Jul 15 05:12:00.524523 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.524500    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-698f8fdbf6-whlhg"]
Jul 15 05:12:00.528583 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:00.528565    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-698f8fdbf6-whlhg"]
Jul 15 05:12:02.387575 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:02.387538    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="253c642c-0f56-4f7c-a2aa-66649df09f64" path="/var/lib/kubelet/pods/253c642c-0f56-4f7c-a2aa-66649df09f64/volumes"
Jul 15 05:12:20.632349 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.632318    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"]
Jul 15 05:12:20.632804 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.632719    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="253c642c-0f56-4f7c-a2aa-66649df09f64" containerName="manager"
Jul 15 05:12:20.632804 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.632732    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="253c642c-0f56-4f7c-a2aa-66649df09f64" containerName="manager"
Jul 15 05:12:20.632804 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.632796    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="253c642c-0f56-4f7c-a2aa-66649df09f64" containerName="manager"
Jul 15 05:12:20.635093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.635075    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.637947 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.637927    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dbgk5\""
Jul 15 05:12:20.638056 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.638031    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 05:12:20.649302 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.649279    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"]
Jul 15 05:12:20.769144 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.769100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74accdc7-b5f4-4b56-8c46-232ee4dbf183-openshift-service-ca\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.769144 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.769147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbkx2\" (UniqueName: \"kubernetes.io/projected/74accdc7-b5f4-4b56-8c46-232ee4dbf183-kube-api-access-lbkx2\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.769396 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.769270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74accdc7-b5f4-4b56-8c46-232ee4dbf183-maas-api-tls\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.870005 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.869973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74accdc7-b5f4-4b56-8c46-232ee4dbf183-openshift-service-ca\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.870005 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.870008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbkx2\" (UniqueName: \"kubernetes.io/projected/74accdc7-b5f4-4b56-8c46-232ee4dbf183-kube-api-access-lbkx2\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.870187 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.870042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74accdc7-b5f4-4b56-8c46-232ee4dbf183-maas-api-tls\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.870187 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:12:20.870138    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 15 05:12:20.870268 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:12:20.870220    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74accdc7-b5f4-4b56-8c46-232ee4dbf183-maas-api-tls podName:74accdc7-b5f4-4b56-8c46-232ee4dbf183 nodeName:}" failed. No retries permitted until 2026-07-15 05:12:21.370187785 +0000 UTC m=+819.537784435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74accdc7-b5f4-4b56-8c46-232ee4dbf183-maas-api-tls") pod "maas-api-6b456cc7c-wdh64" (UID: "74accdc7-b5f4-4b56-8c46-232ee4dbf183") : secret "maas-api-serving-cert" not found
Jul 15 05:12:20.870620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.870601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74accdc7-b5f4-4b56-8c46-232ee4dbf183-openshift-service-ca\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:20.884815 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:20.884746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbkx2\" (UniqueName: \"kubernetes.io/projected/74accdc7-b5f4-4b56-8c46-232ee4dbf183-kube-api-access-lbkx2\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:21.171652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.171581    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"]
Jul 15 05:12:21.174474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.174361    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.177084 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.177063    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 05:12:21.177243 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.177124    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sr7hd\""
Jul 15 05:12:21.188320 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.188298    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"]
Jul 15 05:12:21.219141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.219116    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"]
Jul 15 05:12:21.223498 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.223474    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.230707 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.230683    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"]
Jul 15 05:12:21.273557 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.273528    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb2c09a-a467-4797-8be0-facd56b59937-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-vbn7w\" (UID: \"7bb2c09a-a467-4797-8be0-facd56b59937\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.273668 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.273563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996fd7c1-5f9d-467e-a260-622a64156bd8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-s9n2p\" (UID: \"996fd7c1-5f9d-467e-a260-622a64156bd8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.273668 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.273614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvfbn\" (UniqueName: \"kubernetes.io/projected/7bb2c09a-a467-4797-8be0-facd56b59937-kube-api-access-dvfbn\") pod \"payload-pre-processing-74584bb9f8-vbn7w\" (UID: \"7bb2c09a-a467-4797-8be0-facd56b59937\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.273762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.273694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwggf\" (UniqueName: \"kubernetes.io/projected/996fd7c1-5f9d-467e-a260-622a64156bd8-kube-api-access-vwggf\") pod \"payload-processing-f9b8d9b9b-s9n2p\" (UID: \"996fd7c1-5f9d-467e-a260-622a64156bd8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.374775 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.374736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwggf\" (UniqueName: \"kubernetes.io/projected/996fd7c1-5f9d-467e-a260-622a64156bd8-kube-api-access-vwggf\") pod \"payload-processing-f9b8d9b9b-s9n2p\" (UID: \"996fd7c1-5f9d-467e-a260-622a64156bd8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.374956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.374802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb2c09a-a467-4797-8be0-facd56b59937-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-vbn7w\" (UID: \"7bb2c09a-a467-4797-8be0-facd56b59937\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.374956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.374833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996fd7c1-5f9d-467e-a260-622a64156bd8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-s9n2p\" (UID: \"996fd7c1-5f9d-467e-a260-622a64156bd8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.374956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.374901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvfbn\" (UniqueName: \"kubernetes.io/projected/7bb2c09a-a467-4797-8be0-facd56b59937-kube-api-access-dvfbn\") pod \"payload-pre-processing-74584bb9f8-vbn7w\" (UID: \"7bb2c09a-a467-4797-8be0-facd56b59937\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.374956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.374942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74accdc7-b5f4-4b56-8c46-232ee4dbf183-maas-api-tls\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:21.376164 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.376135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb2c09a-a467-4797-8be0-facd56b59937-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-vbn7w\" (UID: \"7bb2c09a-a467-4797-8be0-facd56b59937\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.377275 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.377035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/996fd7c1-5f9d-467e-a260-622a64156bd8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-s9n2p\" (UID: \"996fd7c1-5f9d-467e-a260-622a64156bd8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.378183 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.378155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74accdc7-b5f4-4b56-8c46-232ee4dbf183-maas-api-tls\") pod \"maas-api-6b456cc7c-wdh64\" (UID: \"74accdc7-b5f4-4b56-8c46-232ee4dbf183\") " pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:21.388939 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.388893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvfbn\" (UniqueName: \"kubernetes.io/projected/7bb2c09a-a467-4797-8be0-facd56b59937-kube-api-access-dvfbn\") pod \"payload-pre-processing-74584bb9f8-vbn7w\" (UID: \"7bb2c09a-a467-4797-8be0-facd56b59937\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.390728 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.390685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwggf\" (UniqueName: \"kubernetes.io/projected/996fd7c1-5f9d-467e-a260-622a64156bd8-kube-api-access-vwggf\") pod \"payload-processing-f9b8d9b9b-s9n2p\" (UID: \"996fd7c1-5f9d-467e-a260-622a64156bd8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.486110 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.486024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:21.538234 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.537505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:21.545614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.545465    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:21.655126 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.655098    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"]
Jul 15 05:12:21.657265 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:12:21.657236    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996fd7c1_5f9d_467e_a260_622a64156bd8.slice/crio-6014a8bc9ee7a93fb257e2cd97c87a1eed8a6eb9368005574caf16e8b7f21865 WatchSource:0}: Error finding container 6014a8bc9ee7a93fb257e2cd97c87a1eed8a6eb9368005574caf16e8b7f21865: Status 404 returned error can't find the container with id 6014a8bc9ee7a93fb257e2cd97c87a1eed8a6eb9368005574caf16e8b7f21865
Jul 15 05:12:21.710827 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.710786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"]
Jul 15 05:12:21.712498 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:12:21.712469    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb2c09a_a467_4797_8be0_facd56b59937.slice/crio-685b8a9cca64cca13a610dfda4fec4a28026c960eb50acac3d75d8250e25d308 WatchSource:0}: Error finding container 685b8a9cca64cca13a610dfda4fec4a28026c960eb50acac3d75d8250e25d308: Status 404 returned error can't find the container with id 685b8a9cca64cca13a610dfda4fec4a28026c960eb50acac3d75d8250e25d308
Jul 15 05:12:21.725178 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:21.725159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"]
Jul 15 05:12:21.727110 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:12:21.727078    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74accdc7_b5f4_4b56_8c46_232ee4dbf183.slice/crio-220cf2908afd9d58dfa667c7e45535f65a31e0a5f2a86a7a9cb3e4551f141fd6 WatchSource:0}: Error finding container 220cf2908afd9d58dfa667c7e45535f65a31e0a5f2a86a7a9cb3e4551f141fd6: Status 404 returned error can't find the container with id 220cf2908afd9d58dfa667c7e45535f65a31e0a5f2a86a7a9cb3e4551f141fd6
Jul 15 05:12:22.631402 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:22.631335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64" event={"ID":"74accdc7-b5f4-4b56-8c46-232ee4dbf183","Type":"ContainerStarted","Data":"220cf2908afd9d58dfa667c7e45535f65a31e0a5f2a86a7a9cb3e4551f141fd6"}
Jul 15 05:12:22.633048 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:22.632970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w" event={"ID":"7bb2c09a-a467-4797-8be0-facd56b59937","Type":"ContainerStarted","Data":"685b8a9cca64cca13a610dfda4fec4a28026c960eb50acac3d75d8250e25d308"}
Jul 15 05:12:22.634705 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:22.634652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p" event={"ID":"996fd7c1-5f9d-467e-a260-622a64156bd8","Type":"ContainerStarted","Data":"6014a8bc9ee7a93fb257e2cd97c87a1eed8a6eb9368005574caf16e8b7f21865"}
Jul 15 05:12:24.645981 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.645946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w" event={"ID":"7bb2c09a-a467-4797-8be0-facd56b59937","Type":"ContainerStarted","Data":"6a0d738f9a5772a0bdbd445c96dc482ecb49545f15100d8d9e916b51ee52ffa7"}
Jul 15 05:12:24.646442 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.646022    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:24.647495 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.647461    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p" event={"ID":"996fd7c1-5f9d-467e-a260-622a64156bd8","Type":"ContainerStarted","Data":"4220c216a0af54c1397b9e136d17e1933029ad520ab15b59e8bf739244ebacee"}
Jul 15 05:12:24.647612 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.647516    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:24.648817 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.648800    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64" event={"ID":"74accdc7-b5f4-4b56-8c46-232ee4dbf183","Type":"ContainerStarted","Data":"64cbf6357b6e5c774e131971dce30b5f18038559f6d39cd7b27580c8d1deca46"}
Jul 15 05:12:24.648913 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.648902    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:24.662464 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.662410    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w" podStartSLOduration=1.448831664 podStartE2EDuration="3.662394338s" totalImagesPullingTime="2.213562674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:12:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:12:24.661436672 +0000 UTC m=+822.829033357" watchObservedRunningTime="2026-07-15 05:12:24.662394338 +0000 UTC m=+822.829991011"
Jul 15 05:12:24.678616 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.678566    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p" podStartSLOduration=1.412374189 podStartE2EDuration="3.678549074s" totalImagesPullingTime="2.266174885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:12:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:12:24.676381826 +0000 UTC m=+822.843978499" watchObservedRunningTime="2026-07-15 05:12:24.678549074 +0000 UTC m=+822.846145742"
Jul 15 05:12:24.694137 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:24.694087    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64" podStartSLOduration=2.49670175 podStartE2EDuration="4.694070339s" totalImagesPullingTime="2.197368589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:12:24.692011227 +0000 UTC m=+822.859607900" watchObservedRunningTime="2026-07-15 05:12:24.694070339 +0000 UTC m=+822.861667013"
Jul 15 05:12:29.673248 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:29.673190    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b2078b7-4b31-42c0-b440-ccfe7c196996" containerID="f97ae2e481842f64334a11724ac0691cb2bcf742777e5065af70cb893c9becdd" exitCode=0
Jul 15 05:12:29.673646 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:29.673271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" event={"ID":"2b2078b7-4b31-42c0-b440-ccfe7c196996","Type":"ContainerDied","Data":"f97ae2e481842f64334a11724ac0691cb2bcf742777e5065af70cb893c9becdd"}
Jul 15 05:12:29.673646 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:29.673620    2581 scope.go:117] "RemoveContainer" containerID="f97ae2e481842f64334a11724ac0691cb2bcf742777e5065af70cb893c9becdd"
Jul 15 05:12:29.674631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:29.674598    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:12:29.674704 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:29.674686    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:12:29.674757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:29.674740    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:12:30.659965 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:30.659938    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b456cc7c-wdh64"
Jul 15 05:12:30.680125 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:30.680095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" event={"ID":"2b2078b7-4b31-42c0-b440-ccfe7c196996","Type":"ContainerStarted","Data":"714414916e92dd2fadcb0349e526bd3e6174cf34bbe2716a5ec7c2804dc52ff0"}
Jul 15 05:12:30.705231 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:30.705184    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:12:30.705402 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:30.705362    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6czc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 15 05:12:30.705473 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:30.705444    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" podUID="2b2078b7-4b31-42c0-b440-ccfe7c196996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 15 05:12:31.705174 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:31.705133    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6czc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 15 05:12:31.705578 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:31.705192    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" podUID="2b2078b7-4b31-42c0-b440-ccfe7c196996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 15 05:12:32.705272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:32.705227    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6czc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 15 05:12:32.705720 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:32.705314    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" podUID="2b2078b7-4b31-42c0-b440-ccfe7c196996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 15 05:12:33.704738 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:33.704702    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6czc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 15 05:12:33.704912 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:33.704767    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" podUID="2b2078b7-4b31-42c0-b440-ccfe7c196996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 15 05:12:34.704675 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:34.704640    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6czc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 15 05:12:34.705037 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:34.704706    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5" podUID="2b2078b7-4b31-42c0-b440-ccfe7c196996" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 15 05:12:35.655931 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:35.655902    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-s9n2p"
Jul 15 05:12:35.656127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:35.656031    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-vbn7w"
Jul 15 05:12:35.709460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:35.709427    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:12:35.709871 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:35.709763    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:12:35.710579 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:12:35.710557    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6czc5"
Jul 15 05:13:16.933528 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:16.933494    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"]
Jul 15 05:13:16.937386 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:16.937364    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:16.940132 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:16.940112    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zcd2w\""
Jul 15 05:13:16.940250 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:16.940112    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 05:13:16.940250 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:16.940112    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 05:13:16.941381 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:16.941364    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 15 05:13:16.946978 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:16.946956    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"]
Jul 15 05:13:17.058377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.058354    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.058502 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.058395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.058542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.058493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.058542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.058519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h97j4\" (UniqueName: \"kubernetes.io/projected/1da49ef6-2a86-4512-a219-b9dda647a939-kube-api-access-h97j4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.058611 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.058554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da49ef6-2a86-4512-a219-b9dda647a939-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.058611 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.058572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.159694 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.159662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.159694 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.159695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h97j4\" (UniqueName: \"kubernetes.io/projected/1da49ef6-2a86-4512-a219-b9dda647a939-kube-api-access-h97j4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.160191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.159726    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da49ef6-2a86-4512-a219-b9dda647a939-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.160191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.159744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.160191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.159785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.160191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.159813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.160191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.160125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.160191 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.160162    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.160483 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.160193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.161997 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.161976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da49ef6-2a86-4512-a219-b9dda647a939-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.162192 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.162175    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da49ef6-2a86-4512-a219-b9dda647a939-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.167953 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.167933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h97j4\" (UniqueName: \"kubernetes.io/projected/1da49ef6-2a86-4512-a219-b9dda647a939-kube-api-access-h97j4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gb2m8\" (UID: \"1da49ef6-2a86-4512-a219-b9dda647a939\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.248277 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.248250    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:17.379087 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.379067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"]
Jul 15 05:13:17.381440 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:13:17.381412    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da49ef6_2a86_4512_a219_b9dda647a939.slice/crio-ce4816b9ecc51640dc1a4fd0093c18abe8e82159a11189ecdd55815e25d9f0c8 WatchSource:0}: Error finding container ce4816b9ecc51640dc1a4fd0093c18abe8e82159a11189ecdd55815e25d9f0c8: Status 404 returned error can't find the container with id ce4816b9ecc51640dc1a4fd0093c18abe8e82159a11189ecdd55815e25d9f0c8
Jul 15 05:13:17.919059 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:17.919015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8" event={"ID":"1da49ef6-2a86-4512-a219-b9dda647a939","Type":"ContainerStarted","Data":"ce4816b9ecc51640dc1a4fd0093c18abe8e82159a11189ecdd55815e25d9f0c8"}
Jul 15 05:13:18.926053 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:18.926014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8" event={"ID":"1da49ef6-2a86-4512-a219-b9dda647a939","Type":"ContainerStarted","Data":"b04052fa11e343dbff6aa50a4542823dbaac9372955ceb81dfc11165481f586b"}
Jul 15 05:13:18.944231 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:18.944172    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8" podStartSLOduration=1.774164999 podStartE2EDuration="2.944158466s" totalImagesPullingTime="1.169993467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:13:18.94337804 +0000 UTC m=+877.110974712" watchObservedRunningTime="2026-07-15 05:13:18.944158466 +0000 UTC m=+877.111755138"
Jul 15 05:13:27.248557 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:27.248525    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:27.248920 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:27.248582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:27.260977 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:27.260952    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:27.987327 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:27.987293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gb2m8"
Jul 15 05:13:39.438562 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.438459    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"]
Jul 15 05:13:39.444027 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.444000    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.446860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.446833    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 15 05:13:39.451652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.451623    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"]
Jul 15 05:13:39.548489 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.548461    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jtth\" (UniqueName: \"kubernetes.io/projected/8f5eb2cc-61d1-4f67-8d58-53a182263b80-kube-api-access-4jtth\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.548633 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.548511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.548633 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.548597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5eb2cc-61d1-4f67-8d58-53a182263b80-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.548734 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.548642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.548734 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.548716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.548825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.548779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650084 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jtth\" (UniqueName: \"kubernetes.io/projected/8f5eb2cc-61d1-4f67-8d58-53a182263b80-kube-api-access-4jtth\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5eb2cc-61d1-4f67-8d58-53a182263b80-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650524 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650583 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650583 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.650721 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.650695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.652366 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.652345    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5eb2cc-61d1-4f67-8d58-53a182263b80-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.652594 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.652575    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5eb2cc-61d1-4f67-8d58-53a182263b80-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.658916 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.658892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jtth\" (UniqueName: \"kubernetes.io/projected/8f5eb2cc-61d1-4f67-8d58-53a182263b80-kube-api-access-4jtth\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxfbq\" (UID: \"8f5eb2cc-61d1-4f67-8d58-53a182263b80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.778431 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.778338    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:39.949845 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:39.949807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"]
Jul 15 05:13:40.030489 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:40.030397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq" event={"ID":"8f5eb2cc-61d1-4f67-8d58-53a182263b80","Type":"ContainerStarted","Data":"09c356dee8b65d123ae5e0cd65b9efec30307c6b8c22b9221662b9387a9e5594"}
Jul 15 05:13:41.036773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:41.036734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq" event={"ID":"8f5eb2cc-61d1-4f67-8d58-53a182263b80","Type":"ContainerStarted","Data":"922d2dd1f7e703c5fae42bae159c82a15214af979cae163771eca286cdf0b3d2"}
Jul 15 05:13:41.054731 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:41.054687    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq" podStartSLOduration=1.841779239 podStartE2EDuration="2.05467023s" totalImagesPullingTime="212.890991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:13:41.054005988 +0000 UTC m=+899.221602663" watchObservedRunningTime="2026-07-15 05:13:41.05467023 +0000 UTC m=+899.222266898"
Jul 15 05:13:49.779409 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:49.779360    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:49.780004 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:49.779469    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:49.792326 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:49.792298    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:13:50.092152 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:13:50.092127    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxfbq"
Jul 15 05:14:46.134214 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.134165    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8645cb74c8-w9k5t"]
Jul 15 05:14:46.137441 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.137423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.145706 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.145686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8645cb74c8-w9k5t"]
Jul 15 05:14:46.195749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.195721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xflt2\" (UniqueName: \"kubernetes.io/projected/1716f25f-6c15-460f-ae10-bb47ce97579a-kube-api-access-xflt2\") pod \"authorino-8645cb74c8-w9k5t\" (UID: \"1716f25f-6c15-460f-ae10-bb47ce97579a\") " pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.195852 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.195789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1716f25f-6c15-460f-ae10-bb47ce97579a-tls-cert\") pod \"authorino-8645cb74c8-w9k5t\" (UID: \"1716f25f-6c15-460f-ae10-bb47ce97579a\") " pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.296833 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.296804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1716f25f-6c15-460f-ae10-bb47ce97579a-tls-cert\") pod \"authorino-8645cb74c8-w9k5t\" (UID: \"1716f25f-6c15-460f-ae10-bb47ce97579a\") " pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.296935 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.296873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xflt2\" (UniqueName: \"kubernetes.io/projected/1716f25f-6c15-460f-ae10-bb47ce97579a-kube-api-access-xflt2\") pod \"authorino-8645cb74c8-w9k5t\" (UID: \"1716f25f-6c15-460f-ae10-bb47ce97579a\") " pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.299531 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.299508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1716f25f-6c15-460f-ae10-bb47ce97579a-tls-cert\") pod \"authorino-8645cb74c8-w9k5t\" (UID: \"1716f25f-6c15-460f-ae10-bb47ce97579a\") " pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.305174 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.305148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xflt2\" (UniqueName: \"kubernetes.io/projected/1716f25f-6c15-460f-ae10-bb47ce97579a-kube-api-access-xflt2\") pod \"authorino-8645cb74c8-w9k5t\" (UID: \"1716f25f-6c15-460f-ae10-bb47ce97579a\") " pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.446763 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.446694    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8645cb74c8-w9k5t"
Jul 15 05:14:46.582536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.582423    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8645cb74c8-w9k5t"]
Jul 15 05:14:46.585311 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:14:46.585275    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1716f25f_6c15_460f_ae10_bb47ce97579a.slice/crio-7c572124d8cad790e44adf0bdde595970124999d94a7c66c4a4d2e45163feb26 WatchSource:0}: Error finding container 7c572124d8cad790e44adf0bdde595970124999d94a7c66c4a4d2e45163feb26: Status 404 returned error can't find the container with id 7c572124d8cad790e44adf0bdde595970124999d94a7c66c4a4d2e45163feb26
Jul 15 05:14:46.586585 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:46.586568    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:14:47.358479 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.358083    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8645cb74c8-w9k5t" event={"ID":"1716f25f-6c15-460f-ae10-bb47ce97579a","Type":"ContainerStarted","Data":"405d3b4ec5e706e1e63eff4a4c5527e91e976dc6dd4dc8d4ab69f2509b6c8fe5"}
Jul 15 05:14:47.358479 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.358443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8645cb74c8-w9k5t" event={"ID":"1716f25f-6c15-460f-ae10-bb47ce97579a","Type":"ContainerStarted","Data":"7c572124d8cad790e44adf0bdde595970124999d94a7c66c4a4d2e45163feb26"}
Jul 15 05:14:47.377823 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.377771    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8645cb74c8-w9k5t" podStartSLOduration=0.901944742 podStartE2EDuration="1.377750369s" totalImagesPullingTime="475.805627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:14:47.375152004 +0000 UTC m=+965.542748698" watchObservedRunningTime="2026-07-15 05:14:47.377750369 +0000 UTC m=+965.545347042"
Jul 15 05:14:47.414499 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.414456    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-595bcff787-t48cd"]
Jul 15 05:14:47.414743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.414714    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-595bcff787-t48cd" podUID="e6fae417-ed08-407d-91af-bfcabee47f33" containerName="authorino" containerID="cri-o://773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93" gracePeriod=30
Jul 15 05:14:47.691309 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.691280    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:14:47.811271 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.811236    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6fae417-ed08-407d-91af-bfcabee47f33-tls-cert\") pod \"e6fae417-ed08-407d-91af-bfcabee47f33\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") "
Jul 15 05:14:47.811473 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.811307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmxbm\" (UniqueName: \"kubernetes.io/projected/e6fae417-ed08-407d-91af-bfcabee47f33-kube-api-access-gmxbm\") pod \"e6fae417-ed08-407d-91af-bfcabee47f33\" (UID: \"e6fae417-ed08-407d-91af-bfcabee47f33\") "
Jul 15 05:14:47.813596 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.813567    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6fae417-ed08-407d-91af-bfcabee47f33-kube-api-access-gmxbm" (OuterVolumeSpecName: "kube-api-access-gmxbm") pod "e6fae417-ed08-407d-91af-bfcabee47f33" (UID: "e6fae417-ed08-407d-91af-bfcabee47f33"). InnerVolumeSpecName "kube-api-access-gmxbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:14:47.824800 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.824772    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6fae417-ed08-407d-91af-bfcabee47f33-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e6fae417-ed08-407d-91af-bfcabee47f33" (UID: "e6fae417-ed08-407d-91af-bfcabee47f33"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:14:47.912666 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.912625    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6fae417-ed08-407d-91af-bfcabee47f33-tls-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:14:47.912666 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:47.912658    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmxbm\" (UniqueName: \"kubernetes.io/projected/e6fae417-ed08-407d-91af-bfcabee47f33-kube-api-access-gmxbm\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:14:48.364497 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.364457    2581 generic.go:356] "Generic (PLEG): container finished" podID="e6fae417-ed08-407d-91af-bfcabee47f33" containerID="773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93" exitCode=0
Jul 15 05:14:48.364898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.364506    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595bcff787-t48cd"
Jul 15 05:14:48.364898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.364545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595bcff787-t48cd" event={"ID":"e6fae417-ed08-407d-91af-bfcabee47f33","Type":"ContainerDied","Data":"773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93"}
Jul 15 05:14:48.364898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.364587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595bcff787-t48cd" event={"ID":"e6fae417-ed08-407d-91af-bfcabee47f33","Type":"ContainerDied","Data":"4507c8f3c971aa512cd208a0a911cd9c3c62e80e7f35f7be84404f63a003c31c"}
Jul 15 05:14:48.364898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.364608    2581 scope.go:117] "RemoveContainer" containerID="773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93"
Jul 15 05:14:48.375601 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.375556    2581 scope.go:117] "RemoveContainer" containerID="773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93"
Jul 15 05:14:48.375988 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:14:48.375961    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93\": container with ID starting with 773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93 not found: ID does not exist" containerID="773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93"
Jul 15 05:14:48.376082 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.375999    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93"} err="failed to get container status \"773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93\": rpc error: code = NotFound desc = could not find container \"773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93\": container with ID starting with 773edc8e4e19ffaf6feb3c340f10841ebd2dbbb9091377915b8e28594b90cf93 not found: ID does not exist"
Jul 15 05:14:48.391351 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.391325    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-595bcff787-t48cd"]
Jul 15 05:14:48.405137 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:48.405112    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-595bcff787-t48cd"]
Jul 15 05:14:50.387612 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:14:50.387577    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6fae417-ed08-407d-91af-bfcabee47f33" path="/var/lib/kubelet/pods/e6fae417-ed08-407d-91af-bfcabee47f33/volumes"
Jul 15 05:15:00.143431 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.143398    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6"]
Jul 15 05:15:00.143797 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.143790    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6fae417-ed08-407d-91af-bfcabee47f33" containerName="authorino"
Jul 15 05:15:00.143840 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.143800    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6fae417-ed08-407d-91af-bfcabee47f33" containerName="authorino"
Jul 15 05:15:00.143875 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.143868    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6fae417-ed08-407d-91af-bfcabee47f33" containerName="authorino"
Jul 15 05:15:00.148363 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.148345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6"
Jul 15 05:15:00.150999 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.150971    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2zt2\""
Jul 15 05:15:00.153903 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.153880    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6"]
Jul 15 05:15:00.160341 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.160324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6"
Jul 15 05:15:00.294090 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.294053    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6"]
Jul 15 05:15:00.296359 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:15:00.296331    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf852e8c6_9276_4e30_b240_6403daebaaab.slice/crio-87dc9b18cad5b176527fc4cbe1c876ee9ba71d446f48b42811ec2f81b3910def WatchSource:0}: Error finding container 87dc9b18cad5b176527fc4cbe1c876ee9ba71d446f48b42811ec2f81b3910def: Status 404 returned error can't find the container with id 87dc9b18cad5b176527fc4cbe1c876ee9ba71d446f48b42811ec2f81b3910def
Jul 15 05:15:00.430654 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:00.430586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6" event={"ID":"f852e8c6-9276-4e30-b240-6403daebaaab","Type":"ContainerStarted","Data":"87dc9b18cad5b176527fc4cbe1c876ee9ba71d446f48b42811ec2f81b3910def"}
Jul 15 05:15:03.448694 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:03.448657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6" event={"ID":"f852e8c6-9276-4e30-b240-6403daebaaab","Type":"ContainerStarted","Data":"b3e761399b99a977adeb37c013b8a1ded461a9c987356943a92543b4cc9d148c"}
Jul 15 05:15:03.464924 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:03.464878    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6" podStartSLOduration=1.191894081 podStartE2EDuration="3.464861953s" totalImagesPullingTime="2.272967872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:15:03.462655828 +0000 UTC m=+981.630252502" watchObservedRunningTime="2026-07-15 05:15:03.464861953 +0000 UTC m=+981.632458625"
Jul 15 05:15:04.454896 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:04.454804    2581 generic.go:356] "Generic (PLEG): container finished" podID="f852e8c6-9276-4e30-b240-6403daebaaab" containerID="b3e761399b99a977adeb37c013b8a1ded461a9c987356943a92543b4cc9d148c" exitCode=0
Jul 15 05:15:04.454896 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:04.454872    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6" event={"ID":"f852e8c6-9276-4e30-b240-6403daebaaab","Type":"ContainerDied","Data":"b3e761399b99a977adeb37c013b8a1ded461a9c987356943a92543b4cc9d148c"}
Jul 15 05:15:05.594106 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:05.594081    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6"
Jul 15 05:15:06.466175 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:06.466148    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6"
Jul 15 05:15:06.466341 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:06.466146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734875-s5dp6" event={"ID":"f852e8c6-9276-4e30-b240-6403daebaaab","Type":"ContainerDied","Data":"87dc9b18cad5b176527fc4cbe1c876ee9ba71d446f48b42811ec2f81b3910def"}
Jul 15 05:15:06.466341 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:15:06.466270    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87dc9b18cad5b176527fc4cbe1c876ee9ba71d446f48b42811ec2f81b3910def"
Jul 15 05:16:14.002006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.001979    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77f5789774-b5wh6"]
Jul 15 05:16:14.002493 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.002222    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77f5789774-b5wh6" podUID="ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" containerName="manager" containerID="cri-o://78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322" gracePeriod=30
Jul 15 05:16:14.258895 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.258839    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:16:14.365182 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.365151    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cls4\" (UniqueName: \"kubernetes.io/projected/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-kube-api-access-5cls4\") pod \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") "
Jul 15 05:16:14.365371 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.365195    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-webhook-cert\") pod \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\" (UID: \"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92\") "
Jul 15 05:16:14.367289 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.367251    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-kube-api-access-5cls4" (OuterVolumeSpecName: "kube-api-access-5cls4") pod "ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" (UID: "ba2686e9-5aae-44c9-8ec7-5f94f74bbd92"). InnerVolumeSpecName "kube-api-access-5cls4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:16:14.367407 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.367289    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" (UID: "ba2686e9-5aae-44c9-8ec7-5f94f74bbd92"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:16:14.466111 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.466082    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-webhook-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:16:14.466111 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.466106    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cls4\" (UniqueName: \"kubernetes.io/projected/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92-kube-api-access-5cls4\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:16:14.819103 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.819069    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" containerID="78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322" exitCode=0
Jul 15 05:16:14.819283 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.819116    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f5789774-b5wh6"
Jul 15 05:16:14.819283 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.819108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f5789774-b5wh6" event={"ID":"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92","Type":"ContainerDied","Data":"78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322"}
Jul 15 05:16:14.819283 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.819241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f5789774-b5wh6" event={"ID":"ba2686e9-5aae-44c9-8ec7-5f94f74bbd92","Type":"ContainerDied","Data":"df15139698067be9c67ea510ceac53ef56646efdf80de8233e6790feb22b74d9"}
Jul 15 05:16:14.819283 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.819266    2581 scope.go:117] "RemoveContainer" containerID="78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322"
Jul 15 05:16:14.828133 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.828116    2581 scope.go:117] "RemoveContainer" containerID="78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322"
Jul 15 05:16:14.828391 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:16:14.828362    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322\": container with ID starting with 78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322 not found: ID does not exist" containerID="78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322"
Jul 15 05:16:14.828439 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.828399    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322"} err="failed to get container status \"78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322\": rpc error: code = NotFound desc = could not find container \"78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322\": container with ID starting with 78874034488aca2fde82f1896e98fd425805220a7eca9ef578302c6f95741322 not found: ID does not exist"
Jul 15 05:16:14.835240 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.835220    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77f5789774-b5wh6"]
Jul 15 05:16:14.838892 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:14.838873    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77f5789774-b5wh6"]
Jul 15 05:16:15.623674 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.623645    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77f5789774-stv5r"]
Jul 15 05:16:15.624054 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.624039    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" containerName="manager"
Jul 15 05:16:15.624098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.624056    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" containerName="manager"
Jul 15 05:16:15.624098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.624075    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f852e8c6-9276-4e30-b240-6403daebaaab" containerName="cleanup"
Jul 15 05:16:15.624098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.624080    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f852e8c6-9276-4e30-b240-6403daebaaab" containerName="cleanup"
Jul 15 05:16:15.624193 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.624142    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" containerName="manager"
Jul 15 05:16:15.624193 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.624152    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f852e8c6-9276-4e30-b240-6403daebaaab" containerName="cleanup"
Jul 15 05:16:15.628421 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.628403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:15.632175 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.632147    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-94hdt\""
Jul 15 05:16:15.632301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.632219    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 05:16:15.632386 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.632150    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 05:16:15.634707 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.634686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f5789774-stv5r"]
Jul 15 05:16:15.677898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.677870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k86x5\" (UniqueName: \"kubernetes.io/projected/4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8-kube-api-access-k86x5\") pod \"maas-controller-77f5789774-stv5r\" (UID: \"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8\") " pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:15.678009 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.677919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8-webhook-cert\") pod \"maas-controller-77f5789774-stv5r\" (UID: \"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8\") " pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:15.779447 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.779421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k86x5\" (UniqueName: \"kubernetes.io/projected/4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8-kube-api-access-k86x5\") pod \"maas-controller-77f5789774-stv5r\" (UID: \"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8\") " pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:15.779577 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.779470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8-webhook-cert\") pod \"maas-controller-77f5789774-stv5r\" (UID: \"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8\") " pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:15.781829 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.781805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8-webhook-cert\") pod \"maas-controller-77f5789774-stv5r\" (UID: \"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8\") " pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:15.787506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.787486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k86x5\" (UniqueName: \"kubernetes.io/projected/4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8-kube-api-access-k86x5\") pod \"maas-controller-77f5789774-stv5r\" (UID: \"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8\") " pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:15.940400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:15.940293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:16.075216 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:16.075178    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f5789774-stv5r"]
Jul 15 05:16:16.076738 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:16:16.076712    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8ce7c7_e36b_433f_a8ec_6d0d2a0f5ca8.slice/crio-1229a0352082e25896a439fe167443858330d890334d2b1de8177f81213342c1 WatchSource:0}: Error finding container 1229a0352082e25896a439fe167443858330d890334d2b1de8177f81213342c1: Status 404 returned error can't find the container with id 1229a0352082e25896a439fe167443858330d890334d2b1de8177f81213342c1
Jul 15 05:16:16.388869 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:16.388839    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba2686e9-5aae-44c9-8ec7-5f94f74bbd92" path="/var/lib/kubelet/pods/ba2686e9-5aae-44c9-8ec7-5f94f74bbd92/volumes"
Jul 15 05:16:16.832750 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:16.832715    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f5789774-stv5r" event={"ID":"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8","Type":"ContainerStarted","Data":"ebed11e9ae1df664d790fa2d5402bc513f69f7038c7201ad824e81ffb300c4d4"}
Jul 15 05:16:16.833077 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:16.832888    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:16:16.833077 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:16.832908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f5789774-stv5r" event={"ID":"4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8","Type":"ContainerStarted","Data":"1229a0352082e25896a439fe167443858330d890334d2b1de8177f81213342c1"}
Jul 15 05:16:16.855744 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:16.855699    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77f5789774-stv5r" podStartSLOduration=1.471284032 podStartE2EDuration="1.855686044s" totalImagesPullingTime="384.402012ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:16:16.852738934 +0000 UTC m=+1055.020335618" watchObservedRunningTime="2026-07-15 05:16:16.855686044 +0000 UTC m=+1055.023282716"
Jul 15 05:16:27.839641 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:16:27.839607    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77f5789774-stv5r"
Jul 15 05:21:12.969161 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:12.969078    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"]
Jul 15 05:21:12.969696 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:12.969326    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p" podUID="00b909e7-1ddd-4078-8da2-3fccf0ab7209" containerName="manager" containerID="cri-o://622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c" gracePeriod=10
Jul 15 05:21:16.523394 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:16.523369    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:21:16.619426 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:16.619398    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00b909e7-1ddd-4078-8da2-3fccf0ab7209-extensions-socket-volume\") pod \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") "
Jul 15 05:21:16.619577 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:16.619431    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx8cx\" (UniqueName: \"kubernetes.io/projected/00b909e7-1ddd-4078-8da2-3fccf0ab7209-kube-api-access-kx8cx\") pod \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\" (UID: \"00b909e7-1ddd-4078-8da2-3fccf0ab7209\") "
Jul 15 05:21:16.619812 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:16.619791    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b909e7-1ddd-4078-8da2-3fccf0ab7209-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "00b909e7-1ddd-4078-8da2-3fccf0ab7209" (UID: "00b909e7-1ddd-4078-8da2-3fccf0ab7209"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:21:16.621398 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:16.621375    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b909e7-1ddd-4078-8da2-3fccf0ab7209-kube-api-access-kx8cx" (OuterVolumeSpecName: "kube-api-access-kx8cx") pod "00b909e7-1ddd-4078-8da2-3fccf0ab7209" (UID: "00b909e7-1ddd-4078-8da2-3fccf0ab7209"). InnerVolumeSpecName "kube-api-access-kx8cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:21:16.720458 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:16.720434    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00b909e7-1ddd-4078-8da2-3fccf0ab7209-extensions-socket-volume\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:21:16.720458 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:16.720455    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx8cx\" (UniqueName: \"kubernetes.io/projected/00b909e7-1ddd-4078-8da2-3fccf0ab7209-kube-api-access-kx8cx\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:21:17.356860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.356828    2581 generic.go:356] "Generic (PLEG): container finished" podID="00b909e7-1ddd-4078-8da2-3fccf0ab7209" containerID="622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c" exitCode=0
Jul 15 05:21:17.357034 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.356872    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"
Jul 15 05:21:17.357034 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.356912    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p" event={"ID":"00b909e7-1ddd-4078-8da2-3fccf0ab7209","Type":"ContainerDied","Data":"622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c"}
Jul 15 05:21:17.357034 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.356963    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p" event={"ID":"00b909e7-1ddd-4078-8da2-3fccf0ab7209","Type":"ContainerDied","Data":"2b9301feac7e7c08023a4f3a1ce2b786041992ffbaa1b6e547349092b91f29ef"}
Jul 15 05:21:17.357034 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.356987    2581 scope.go:117] "RemoveContainer" containerID="622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c"
Jul 15 05:21:17.371615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.371598    2581 scope.go:117] "RemoveContainer" containerID="622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c"
Jul 15 05:21:17.371966 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:21:17.371944    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c\": container with ID starting with 622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c not found: ID does not exist" containerID="622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c"
Jul 15 05:21:17.372017 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.371976    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c"} err="failed to get container status \"622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c\": rpc error: code = NotFound desc = could not find container \"622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c\": container with ID starting with 622c86b15d42d0309c31f9b4c38a45b61e7b3cc24665b529d3c402461424402c not found: ID does not exist"
Jul 15 05:21:17.383290 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.383261    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"]
Jul 15 05:21:17.385332 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:17.385303    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d6v2p"]
Jul 15 05:21:18.387246 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:21:18.387189    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b909e7-1ddd-4078-8da2-3fccf0ab7209" path="/var/lib/kubelet/pods/00b909e7-1ddd-4078-8da2-3fccf0ab7209/volumes"
Jul 15 05:22:14.171428 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.171394    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"]
Jul 15 05:22:14.171884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.171782    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b909e7-1ddd-4078-8da2-3fccf0ab7209" containerName="manager"
Jul 15 05:22:14.171884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.171794    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b909e7-1ddd-4078-8da2-3fccf0ab7209" containerName="manager"
Jul 15 05:22:14.171884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.171851    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b909e7-1ddd-4078-8da2-3fccf0ab7209" containerName="manager"
Jul 15 05:22:14.175033 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.175017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.177759 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.177740    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p4hm7\""
Jul 15 05:22:14.188753 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.188732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"]
Jul 15 05:22:14.288703 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.288677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx556\" (UniqueName: \"kubernetes.io/projected/2f564665-b8e5-435f-9f98-e127b8f0725c-kube-api-access-tx556\") pod \"kuadrant-operator-controller-manager-86878c769f-tn8bc\" (UID: \"2f564665-b8e5-435f-9f98-e127b8f0725c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.288826 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.288717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f564665-b8e5-435f-9f98-e127b8f0725c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tn8bc\" (UID: \"2f564665-b8e5-435f-9f98-e127b8f0725c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.389894 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.389871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx556\" (UniqueName: \"kubernetes.io/projected/2f564665-b8e5-435f-9f98-e127b8f0725c-kube-api-access-tx556\") pod \"kuadrant-operator-controller-manager-86878c769f-tn8bc\" (UID: \"2f564665-b8e5-435f-9f98-e127b8f0725c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.390048 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.389911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f564665-b8e5-435f-9f98-e127b8f0725c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tn8bc\" (UID: \"2f564665-b8e5-435f-9f98-e127b8f0725c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.390298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.390276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f564665-b8e5-435f-9f98-e127b8f0725c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tn8bc\" (UID: \"2f564665-b8e5-435f-9f98-e127b8f0725c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.400086 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.400064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx556\" (UniqueName: \"kubernetes.io/projected/2f564665-b8e5-435f-9f98-e127b8f0725c-kube-api-access-tx556\") pod \"kuadrant-operator-controller-manager-86878c769f-tn8bc\" (UID: \"2f564665-b8e5-435f-9f98-e127b8f0725c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.484881 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.484809    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:14.620905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.620881    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"]
Jul 15 05:22:14.622539 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:22:14.622511    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f564665_b8e5_435f_9f98_e127b8f0725c.slice/crio-283bef56a6f7027cb75426728c61eed692bfebfb893a7c935d480404c139550d WatchSource:0}: Error finding container 283bef56a6f7027cb75426728c61eed692bfebfb893a7c935d480404c139550d: Status 404 returned error can't find the container with id 283bef56a6f7027cb75426728c61eed692bfebfb893a7c935d480404c139550d
Jul 15 05:22:14.641496 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:14.641455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc" event={"ID":"2f564665-b8e5-435f-9f98-e127b8f0725c","Type":"ContainerStarted","Data":"283bef56a6f7027cb75426728c61eed692bfebfb893a7c935d480404c139550d"}
Jul 15 05:22:15.648220 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:15.648160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc" event={"ID":"2f564665-b8e5-435f-9f98-e127b8f0725c","Type":"ContainerStarted","Data":"0616948f071fa88fb17721237de1f6e3b53c6823e909c4c68a76fccca01b3bd8"}
Jul 15 05:22:15.648629 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:15.648396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:22:15.671614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:15.671569    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc" podStartSLOduration=1.671553731 podStartE2EDuration="1.671553731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:22:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:22:15.668766534 +0000 UTC m=+1413.836363218" watchObservedRunningTime="2026-07-15 05:22:15.671553731 +0000 UTC m=+1413.839150403"
Jul 15 05:22:26.655754 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:22:26.655722    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tn8bc"
Jul 15 05:28:45.867814 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.867784    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"]
Jul 15 05:28:45.871448 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.871429    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.874550 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.874525    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-90c99c81-openshift-default-dockercfg-q28m6\""
Jul 15 05:28:45.888180 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-envoy\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888326 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126fcf13-084a-438a-835c-e3875ee9eef7-istio-podinfo\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888326 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg2tk\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-kube-api-access-rg2tk\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-data\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-istio-token\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-socket\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-certs\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-credential-socket\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.888614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.888591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126fcf13-084a-438a-835c-e3875ee9eef7-istiod-ca-cert\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.897092 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.897071    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"]
Jul 15 05:28:45.989460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126fcf13-084a-438a-835c-e3875ee9eef7-istiod-ca-cert\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.989632 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-envoy\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.989632 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126fcf13-084a-438a-835c-e3875ee9eef7-istio-podinfo\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.989757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg2tk\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-kube-api-access-rg2tk\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.989757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-data\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.989757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-istio-token\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.989757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-socket\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.989757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-certs\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.990012 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-credential-socket\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.990012 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.989962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-data\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.990117 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.990049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-socket\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.990178 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.990142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-credential-socket\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.990263 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.990179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126fcf13-084a-438a-835c-e3875ee9eef7-istiod-ca-cert\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.990263 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.990234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-certs\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.991827 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.991805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-envoy\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.992016 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.991999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126fcf13-084a-438a-835c-e3875ee9eef7-istio-podinfo\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.997739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.997715    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-istio-token\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:45.997838 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:45.997820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg2tk\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-kube-api-access-rg2tk\") pod \"e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") " pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:46.183727 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.183652    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:46.320893 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.320869    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"]
Jul 15 05:28:46.323428 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:28:46.323401    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126fcf13_084a_438a_835c_e3875ee9eef7.slice/crio-3bff40ea30ddc16ac4e013f9a0a469893c09bc41bf6429365d0923a4e44fb60d WatchSource:0}: Error finding container 3bff40ea30ddc16ac4e013f9a0a469893c09bc41bf6429365d0923a4e44fb60d: Status 404 returned error can't find the container with id 3bff40ea30ddc16ac4e013f9a0a469893c09bc41bf6429365d0923a4e44fb60d
Jul 15 05:28:46.325798 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.325762    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:28:46.325895 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.325844    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:28:46.325959 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.325897    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:28:46.655287 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.655251    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" event={"ID":"126fcf13-084a-438a-835c-e3875ee9eef7","Type":"ContainerStarted","Data":"665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32"}
Jul 15 05:28:46.655463 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.655307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" event={"ID":"126fcf13-084a-438a-835c-e3875ee9eef7","Type":"ContainerStarted","Data":"3bff40ea30ddc16ac4e013f9a0a469893c09bc41bf6429365d0923a4e44fb60d"}
Jul 15 05:28:46.677081 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:46.676979    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" podStartSLOduration=1.67696007 podStartE2EDuration="1.67696007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:28:46.674058006 +0000 UTC m=+1804.841654680" watchObservedRunningTime="2026-07-15 05:28:46.67696007 +0000 UTC m=+1804.844556743"
Jul 15 05:28:47.163605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.163573    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"]
Jul 15 05:28:47.167308 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.167285    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.169807 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.169788    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-90c99c81-serving-cert\""
Jul 15 05:28:47.181007 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.180985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"]
Jul 15 05:28:47.184356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.184335    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:47.189874 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.189854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:47.200768 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.200748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93e7fdb-dd34-4201-b032-82c2ffe6941a-openshift-service-ca\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.200866 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.200802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93e7fdb-dd34-4201-b032-82c2ffe6941a-maas-api-tls\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.200866 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.200856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25cb2\" (UniqueName: \"kubernetes.io/projected/b93e7fdb-dd34-4201-b032-82c2ffe6941a-kube-api-access-25cb2\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.301951 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.301914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93e7fdb-dd34-4201-b032-82c2ffe6941a-openshift-service-ca\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.302108 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.301978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93e7fdb-dd34-4201-b032-82c2ffe6941a-maas-api-tls\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.302108 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.302014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25cb2\" (UniqueName: \"kubernetes.io/projected/b93e7fdb-dd34-4201-b032-82c2ffe6941a-kube-api-access-25cb2\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.302666 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.302642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93e7fdb-dd34-4201-b032-82c2ffe6941a-openshift-service-ca\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.304527 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.304497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93e7fdb-dd34-4201-b032-82c2ffe6941a-maas-api-tls\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.310873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.310846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25cb2\" (UniqueName: \"kubernetes.io/projected/b93e7fdb-dd34-4201-b032-82c2ffe6941a-kube-api-access-25cb2\") pod \"maas-api-e2e-ait-90c99c81-57fd55485b-k5jth\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.479738 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.479664    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:47.610709 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.610684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"]
Jul 15 05:28:47.612459 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:28:47.612434    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93e7fdb_dd34_4201_b032_82c2ffe6941a.slice/crio-971605d0b9a3c489b3fce628a7471d6ee0153c26c4e6d26bdd4584228b82257e WatchSource:0}: Error finding container 971605d0b9a3c489b3fce628a7471d6ee0153c26c4e6d26bdd4584228b82257e: Status 404 returned error can't find the container with id 971605d0b9a3c489b3fce628a7471d6ee0153c26c4e6d26bdd4584228b82257e
Jul 15 05:28:47.613857 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.613835    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:28:47.661481 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.661456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth" event={"ID":"b93e7fdb-dd34-4201-b032-82c2ffe6941a","Type":"ContainerStarted","Data":"971605d0b9a3c489b3fce628a7471d6ee0153c26c4e6d26bdd4584228b82257e"}
Jul 15 05:28:47.661803 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.661786    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:47.662659 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:47.662641    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:28:48.667642 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:48.667605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth" event={"ID":"b93e7fdb-dd34-4201-b032-82c2ffe6941a","Type":"ContainerStarted","Data":"fba174e37ca1edecf3886613bccc3e5a597169f6f550e82a05f22000037d2136"}
Jul 15 05:28:48.668010 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:48.667857    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:48.683759 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:48.683694    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth" podStartSLOduration=1.239920566 podStartE2EDuration="1.683680384s" totalImagesPullingTime="443.759818ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:28:48.682948035 +0000 UTC m=+1806.850544707" watchObservedRunningTime="2026-07-15 05:28:48.683680384 +0000 UTC m=+1806.851277054"
Jul 15 05:28:54.690610 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:54.690580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:28:58.732873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:58.732847    2581 generic.go:356] "Generic (PLEG): container finished" podID="126fcf13-084a-438a-835c-e3875ee9eef7" containerID="665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32" exitCode=0
Jul 15 05:28:58.733235 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:58.732909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" event={"ID":"126fcf13-084a-438a-835c-e3875ee9eef7","Type":"ContainerDied","Data":"665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32"}
Jul 15 05:28:58.733287 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:58.733279    2581 scope.go:117] "RemoveContainer" containerID="665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32"
Jul 15 05:28:58.734273 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:58.734246    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:28:58.734336 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:58.734320    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:28:58.734375 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:58.734353    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:28:59.740769 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:28:59.740735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" event={"ID":"126fcf13-084a-438a-835c-e3875ee9eef7","Type":"ContainerStarted","Data":"7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e"}
Jul 15 05:29:00.184549 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:00.184515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:29:00.185758 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:00.185728    2581 patch_prober.go:28] interesting pod/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 05:29:00.185973 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:00.185945    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 05:29:01.184775 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:01.184735    2581 patch_prober.go:28] interesting pod/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 05:29:01.185301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:01.184804    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 05:29:02.184227 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:02.184171    2581 patch_prober.go:28] interesting pod/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 05:29:02.184569 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:02.184536    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 05:29:02.662469 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:02.662437    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:29:03.184357 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:03.184322    2581 patch_prober.go:28] interesting pod/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 15 05:29:03.184639 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:03.184612    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 15 05:29:04.188410 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:04.188369    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:29:04.189417 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:04.189398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:29:12.562755 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.562717    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"]
Jul 15 05:29:12.563286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.562954    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth" podUID="b93e7fdb-dd34-4201-b032-82c2ffe6941a" containerName="maas-api" containerID="cri-o://fba174e37ca1edecf3886613bccc3e5a597169f6f550e82a05f22000037d2136" gracePeriod=30
Jul 15 05:29:12.808733 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.808700    2581 generic.go:356] "Generic (PLEG): container finished" podID="b93e7fdb-dd34-4201-b032-82c2ffe6941a" containerID="fba174e37ca1edecf3886613bccc3e5a597169f6f550e82a05f22000037d2136" exitCode=0
Jul 15 05:29:12.808873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.808768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth" event={"ID":"b93e7fdb-dd34-4201-b032-82c2ffe6941a","Type":"ContainerDied","Data":"fba174e37ca1edecf3886613bccc3e5a597169f6f550e82a05f22000037d2136"}
Jul 15 05:29:12.826269 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.826193    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:29:12.931700 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.931674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93e7fdb-dd34-4201-b032-82c2ffe6941a-openshift-service-ca\") pod \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") "
Jul 15 05:29:12.931844 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.931774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25cb2\" (UniqueName: \"kubernetes.io/projected/b93e7fdb-dd34-4201-b032-82c2ffe6941a-kube-api-access-25cb2\") pod \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") "
Jul 15 05:29:12.931844 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.931802    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93e7fdb-dd34-4201-b032-82c2ffe6941a-maas-api-tls\") pod \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\" (UID: \"b93e7fdb-dd34-4201-b032-82c2ffe6941a\") "
Jul 15 05:29:12.932086 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.932060    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93e7fdb-dd34-4201-b032-82c2ffe6941a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b93e7fdb-dd34-4201-b032-82c2ffe6941a" (UID: "b93e7fdb-dd34-4201-b032-82c2ffe6941a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:29:12.933926 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.933901    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93e7fdb-dd34-4201-b032-82c2ffe6941a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b93e7fdb-dd34-4201-b032-82c2ffe6941a" (UID: "b93e7fdb-dd34-4201-b032-82c2ffe6941a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:29:12.934040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:12.933941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93e7fdb-dd34-4201-b032-82c2ffe6941a-kube-api-access-25cb2" (OuterVolumeSpecName: "kube-api-access-25cb2") pod "b93e7fdb-dd34-4201-b032-82c2ffe6941a" (UID: "b93e7fdb-dd34-4201-b032-82c2ffe6941a"). InnerVolumeSpecName "kube-api-access-25cb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:29:13.032750 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.032718    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25cb2\" (UniqueName: \"kubernetes.io/projected/b93e7fdb-dd34-4201-b032-82c2ffe6941a-kube-api-access-25cb2\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:13.032750 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.032744    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93e7fdb-dd34-4201-b032-82c2ffe6941a-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:13.032890 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.032758    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93e7fdb-dd34-4201-b032-82c2ffe6941a-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:13.814577 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.814545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth" event={"ID":"b93e7fdb-dd34-4201-b032-82c2ffe6941a","Type":"ContainerDied","Data":"971605d0b9a3c489b3fce628a7471d6ee0153c26c4e6d26bdd4584228b82257e"}
Jul 15 05:29:13.815011 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.814601    2581 scope.go:117] "RemoveContainer" containerID="fba174e37ca1edecf3886613bccc3e5a597169f6f550e82a05f22000037d2136"
Jul 15 05:29:13.815011 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.814553    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"
Jul 15 05:29:13.836161 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.836134    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"]
Jul 15 05:29:13.839707 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:13.839683    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90c99c81-57fd55485b-k5jth"]
Jul 15 05:29:14.387937 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:14.387904    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b93e7fdb-dd34-4201-b032-82c2ffe6941a" path="/var/lib/kubelet/pods/b93e7fdb-dd34-4201-b032-82c2ffe6941a/volumes"
Jul 15 05:29:27.401228 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.401173    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"]
Jul 15 05:29:27.401657 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.401364    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy" containerID="cri-o://7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e" gracePeriod=30
Jul 15 05:29:27.948438 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.948407    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"]
Jul 15 05:29:27.949048 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.949028    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93e7fdb-dd34-4201-b032-82c2ffe6941a" containerName="maas-api"
Jul 15 05:29:27.949134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.949053    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93e7fdb-dd34-4201-b032-82c2ffe6941a" containerName="maas-api"
Jul 15 05:29:27.949170 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.949149    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b93e7fdb-dd34-4201-b032-82c2ffe6941a" containerName="maas-api"
Jul 15 05:29:27.951842 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.951820    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:27.954595 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.954575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ff1de9c2-gw-openshift-default-dockercfg-4dpsb\""
Jul 15 05:29:27.962145 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:27.962123    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"]
Jul 15 05:29:28.058376 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-podinfo\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058546 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-token\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058546 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-data\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058546 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-certs\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058685 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-socket\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058685 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-envoy\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058784 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9p8\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-kube-api-access-7q9p8\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058889 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53cf54f9-ca18-44f2-9272-989c471fe5bf-istiod-ca-cert\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.058942 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.058926    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-credential-socket\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-data\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160478 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160307    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-certs\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160478 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-socket\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160478 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-envoy\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160478 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160432    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9p8\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-kube-api-access-7q9p8\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53cf54f9-ca18-44f2-9272-989c471fe5bf-istiod-ca-cert\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-credential-socket\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-podinfo\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-token\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-certs\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.160711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-data\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.161019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.160946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-credential-socket\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.161019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.161000    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-socket\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.161302 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.161278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53cf54f9-ca18-44f2-9272-989c471fe5bf-istiod-ca-cert\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.163029 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.163012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-envoy\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.163125 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.163107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-podinfo\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.170770 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.170745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9p8\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-kube-api-access-7q9p8\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.171004 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.170980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-token\") pod \"e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") " pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.263970 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.263911    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:28.400518 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.400493    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"]
Jul 15 05:29:28.403249 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:29:28.403218    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53cf54f9_ca18_44f2_9272_989c471fe5bf.slice/crio-188a9dcd685be70d403178008fde8304f02410d648bd7fa3693b6e810a180c21 WatchSource:0}: Error finding container 188a9dcd685be70d403178008fde8304f02410d648bd7fa3693b6e810a180c21: Status 404 returned error can't find the container with id 188a9dcd685be70d403178008fde8304f02410d648bd7fa3693b6e810a180c21
Jul 15 05:29:28.405728 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.405696    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:29:28.405796 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.405755    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:29:28.405796 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.405781    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:29:28.892345 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.892313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" event={"ID":"53cf54f9-ca18-44f2-9272-989c471fe5bf","Type":"ContainerStarted","Data":"e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0"}
Jul 15 05:29:28.892517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.892360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" event={"ID":"53cf54f9-ca18-44f2-9272-989c471fe5bf","Type":"ContainerStarted","Data":"188a9dcd685be70d403178008fde8304f02410d648bd7fa3693b6e810a180c21"}
Jul 15 05:29:28.914301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:28.914253    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podStartSLOduration=1.914238294 podStartE2EDuration="1.914238294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:29:28.913388736 +0000 UTC m=+1847.080985410" watchObservedRunningTime="2026-07-15 05:29:28.914238294 +0000 UTC m=+1847.081834968"
Jul 15 05:29:29.264682 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.264609    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:29.270104 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.270058    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:29.270555 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.270534    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"]
Jul 15 05:29:29.273665 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.273642    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.276197 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.276178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-ff1de9c2-serving-cert\""
Jul 15 05:29:29.282826 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.282803    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"]
Jul 15 05:29:29.372833 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.372803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.372982 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.372837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjws5\" (UniqueName: \"kubernetes.io/projected/2a3f6018-1c07-47a4-ab7c-4503ec12798d-kube-api-access-tjws5\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.373039 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.372984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3f6018-1c07-47a4-ab7c-4503ec12798d-openshift-service-ca\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.473876 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.473837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3f6018-1c07-47a4-ab7c-4503ec12798d-openshift-service-ca\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.474393 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.473890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.474393 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.473910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjws5\" (UniqueName: \"kubernetes.io/projected/2a3f6018-1c07-47a4-ab7c-4503ec12798d-kube-api-access-tjws5\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.474393 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:29:29.474015    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-serving-cert: secret "maas-api-e2e-migrate-ff1de9c2-serving-cert" not found
Jul 15 05:29:29.474393 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:29:29.474079    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls podName:2a3f6018-1c07-47a4-ab7c-4503ec12798d nodeName:}" failed. No retries permitted until 2026-07-15 05:29:29.97405762 +0000 UTC m=+1848.141654289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls") pod "maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj" (UID: "2a3f6018-1c07-47a4-ab7c-4503ec12798d") : secret "maas-api-e2e-migrate-ff1de9c2-serving-cert" not found
Jul 15 05:29:29.474705 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.474680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3f6018-1c07-47a4-ab7c-4503ec12798d-openshift-service-ca\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.482838 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.482814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjws5\" (UniqueName: \"kubernetes.io/projected/2a3f6018-1c07-47a4-ab7c-4503ec12798d-kube-api-access-tjws5\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.897573 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.897546    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:29.898493 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.898471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:29.979663 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.979623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:29.982106 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:29.982076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls\") pod \"maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:30.188114 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:30.188032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:30.324039 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:30.324013    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"]
Jul 15 05:29:30.325546 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:29:30.325520    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3f6018_1c07_47a4_ab7c_4503ec12798d.slice/crio-3cfbe48f8ccb87f0422eebb870bc5593ec9542d1b418fb83ebfc9887b7b4e5a7 WatchSource:0}: Error finding container 3cfbe48f8ccb87f0422eebb870bc5593ec9542d1b418fb83ebfc9887b7b4e5a7: Status 404 returned error can't find the container with id 3cfbe48f8ccb87f0422eebb870bc5593ec9542d1b418fb83ebfc9887b7b4e5a7
Jul 15 05:29:30.904006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:30.903970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj" event={"ID":"2a3f6018-1c07-47a4-ab7c-4503ec12798d","Type":"ContainerStarted","Data":"838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41"}
Jul 15 05:29:30.904445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:30.904030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj" event={"ID":"2a3f6018-1c07-47a4-ab7c-4503ec12798d","Type":"ContainerStarted","Data":"3cfbe48f8ccb87f0422eebb870bc5593ec9542d1b418fb83ebfc9887b7b4e5a7"}
Jul 15 05:29:30.904445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:30.904197    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:30.925236 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:30.925143    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj" podStartSLOduration=1.577071524 podStartE2EDuration="1.925130165s" totalImagesPullingTime="348.058641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:29:30.922443804 +0000 UTC m=+1849.090040500" watchObservedRunningTime="2026-07-15 05:29:30.925130165 +0000 UTC m=+1849.092726837"
Jul 15 05:29:32.661120 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.661064    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:29:32.805885 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.805861    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-socket\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806024 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.805900    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-certs\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806024 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.805923    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-data\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806024 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.805956    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-istio-token\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806024 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.805996    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg2tk\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-kube-api-access-rg2tk\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806022    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126fcf13-084a-438a-835c-e3875ee9eef7-istiod-ca-cert\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806098    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-envoy\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806158    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-credential-socket\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806133    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:29:32.806244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806189    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:29:32.806244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126fcf13-084a-438a-835c-e3875ee9eef7-istio-podinfo\") pod \"126fcf13-084a-438a-835c-e3875ee9eef7\" (UID: \"126fcf13-084a-438a-835c-e3875ee9eef7\") "
Jul 15 05:29:32.806559 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806452    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:29:32.806559 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806523    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126fcf13-084a-438a-835c-e3875ee9eef7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:29:32.806722 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806627    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.806722 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806645    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.806722 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806660    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.806722 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806673    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126fcf13-084a-438a-835c-e3875ee9eef7-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.806932 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.806862    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-data" (OuterVolumeSpecName: "istio-data") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:29:32.808398 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.808366    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/126fcf13-084a-438a-835c-e3875ee9eef7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:29:32.808503 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.808420    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-kube-api-access-rg2tk" (OuterVolumeSpecName: "kube-api-access-rg2tk") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "kube-api-access-rg2tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:29:32.808503 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.808442    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-istio-token" (OuterVolumeSpecName: "istio-token") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:29:32.808503 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.808455    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "126fcf13-084a-438a-835c-e3875ee9eef7" (UID: "126fcf13-084a-438a-835c-e3875ee9eef7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:29:32.907886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.907865    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.907886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.907885    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.908034 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.907893    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg2tk\" (UniqueName: \"kubernetes.io/projected/126fcf13-084a-438a-835c-e3875ee9eef7-kube-api-access-rg2tk\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.908034 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.907903    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126fcf13-084a-438a-835c-e3875ee9eef7-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.908034 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.907911    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126fcf13-084a-438a-835c-e3875ee9eef7-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:29:32.915786 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.915727    2581 generic.go:356] "Generic (PLEG): container finished" podID="126fcf13-084a-438a-835c-e3875ee9eef7" containerID="7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e" exitCode=0
Jul 15 05:29:32.915886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.915786    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"
Jul 15 05:29:32.915886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.915813    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" event={"ID":"126fcf13-084a-438a-835c-e3875ee9eef7","Type":"ContainerDied","Data":"7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e"}
Jul 15 05:29:32.915886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.915867    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg" event={"ID":"126fcf13-084a-438a-835c-e3875ee9eef7","Type":"ContainerDied","Data":"3bff40ea30ddc16ac4e013f9a0a469893c09bc41bf6429365d0923a4e44fb60d"}
Jul 15 05:29:32.915984 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.915889    2581 scope.go:117] "RemoveContainer" containerID="7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e"
Jul 15 05:29:32.924766 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.924753    2581 scope.go:117] "RemoveContainer" containerID="665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32"
Jul 15 05:29:32.932644 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.932627    2581 scope.go:117] "RemoveContainer" containerID="7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e"
Jul 15 05:29:32.932878 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:29:32.932858    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e\": container with ID starting with 7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e not found: ID does not exist" containerID="7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e"
Jul 15 05:29:32.932921 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.932885    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e"} err="failed to get container status \"7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e\": rpc error: code = NotFound desc = could not find container \"7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e\": container with ID starting with 7abecf16055b07d15ed3f1d8b6f9058ba6b190059f32fb27cf9e34b33148c72e not found: ID does not exist"
Jul 15 05:29:32.932921 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.932902    2581 scope.go:117] "RemoveContainer" containerID="665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32"
Jul 15 05:29:32.933124 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:29:32.933107    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32\": container with ID starting with 665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32 not found: ID does not exist" containerID="665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32"
Jul 15 05:29:32.933170 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.933129    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32"} err="failed to get container status \"665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32\": rpc error: code = NotFound desc = could not find container \"665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32\": container with ID starting with 665d69d685d7fa0f966ee402157e1ad2d19673827929ea056ff215cd8f79ed32 not found: ID does not exist"
Jul 15 05:29:32.938311 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.938289    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"]
Jul 15 05:29:32.942576 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:32.942551    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-90c99c81-openshift-default-9bdbbc64c-bfvjg"]
Jul 15 05:29:34.388337 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:34.388260    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" path="/var/lib/kubelet/pods/126fcf13-084a-438a-835c-e3875ee9eef7/volumes"
Jul 15 05:29:36.915018 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:36.914990    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:29:44.899447 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.899414    2581 patch_prober.go:28] interesting pod/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 15 05:29:44.899830 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.899495    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 15 05:29:44.979716 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.979673    2581 generic.go:356] "Generic (PLEG): container finished" podID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerID="e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0" exitCode=0
Jul 15 05:29:44.979851 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.979738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" event={"ID":"53cf54f9-ca18-44f2-9272-989c471fe5bf","Type":"ContainerDied","Data":"e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0"}
Jul 15 05:29:44.980195 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.980175    2581 scope.go:117] "RemoveContainer" containerID="e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0"
Jul 15 05:29:44.981310 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.981277    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:29:44.981427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.981363    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:29:44.981427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:44.981411    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:29:45.993858 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:45.993816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" event={"ID":"53cf54f9-ca18-44f2-9272-989c471fe5bf","Type":"ContainerStarted","Data":"6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e"}
Jul 15 05:29:46.264820 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:46.264729    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:46.265001 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:46.264977    2581 patch_prober.go:28] interesting pod/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 15 05:29:46.265046 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:46.265026    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 15 05:29:47.265162 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:47.265127    2581 patch_prober.go:28] interesting pod/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 15 05:29:47.265602 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:47.265181    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 15 05:29:48.265357 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:48.265319    2581 patch_prober.go:28] interesting pod/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 15 05:29:48.265787 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:48.265378    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 15 05:29:49.264390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:49.264345    2581 patch_prober.go:28] interesting pod/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 15 05:29:49.264562 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:49.264421    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 15 05:29:50.268267 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:50.268236    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:50.268694 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:50.268598    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:29:50.269136 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:29:50.269114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:30:33.893463 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:33.893383    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"]
Jul 15 05:30:33.893961 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:33.893574    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy" containerID="cri-o://6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e" gracePeriod=30
Jul 15 05:30:34.265855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.265771    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"]
Jul 15 05:30:34.266338 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.266280    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj" podUID="2a3f6018-1c07-47a4-ab7c-4503ec12798d" containerName="maas-api" containerID="cri-o://838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41" gracePeriod=30
Jul 15 05:30:34.528759 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.528699    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:30:34.642063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.642031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjws5\" (UniqueName: \"kubernetes.io/projected/2a3f6018-1c07-47a4-ab7c-4503ec12798d-kube-api-access-tjws5\") pod \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") "
Jul 15 05:30:34.642261 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.642101    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls\") pod \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") "
Jul 15 05:30:34.642261 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.642122    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3f6018-1c07-47a4-ab7c-4503ec12798d-openshift-service-ca\") pod \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\" (UID: \"2a3f6018-1c07-47a4-ab7c-4503ec12798d\") "
Jul 15 05:30:34.642526 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.642491    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a3f6018-1c07-47a4-ab7c-4503ec12798d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a3f6018-1c07-47a4-ab7c-4503ec12798d" (UID: "2a3f6018-1c07-47a4-ab7c-4503ec12798d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:30:34.644138 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.644111    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a3f6018-1c07-47a4-ab7c-4503ec12798d-kube-api-access-tjws5" (OuterVolumeSpecName: "kube-api-access-tjws5") pod "2a3f6018-1c07-47a4-ab7c-4503ec12798d" (UID: "2a3f6018-1c07-47a4-ab7c-4503ec12798d"). InnerVolumeSpecName "kube-api-access-tjws5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:30:34.644228 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.644172    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a3f6018-1c07-47a4-ab7c-4503ec12798d" (UID: "2a3f6018-1c07-47a4-ab7c-4503ec12798d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:30:34.743309 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.743283    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a3f6018-1c07-47a4-ab7c-4503ec12798d-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:34.743309 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.743306    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3f6018-1c07-47a4-ab7c-4503ec12798d-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:34.743447 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:34.743316    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjws5\" (UniqueName: \"kubernetes.io/projected/2a3f6018-1c07-47a4-ab7c-4503ec12798d-kube-api-access-tjws5\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:35.259797 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.259764    2581 generic.go:356] "Generic (PLEG): container finished" podID="2a3f6018-1c07-47a4-ab7c-4503ec12798d" containerID="838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41" exitCode=0
Jul 15 05:30:35.260272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.259810    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"
Jul 15 05:30:35.260272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.259850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj" event={"ID":"2a3f6018-1c07-47a4-ab7c-4503ec12798d","Type":"ContainerDied","Data":"838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41"}
Jul 15 05:30:35.260272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.259903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj" event={"ID":"2a3f6018-1c07-47a4-ab7c-4503ec12798d","Type":"ContainerDied","Data":"3cfbe48f8ccb87f0422eebb870bc5593ec9542d1b418fb83ebfc9887b7b4e5a7"}
Jul 15 05:30:35.260272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.259928    2581 scope.go:117] "RemoveContainer" containerID="838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41"
Jul 15 05:30:35.269040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.268996    2581 scope.go:117] "RemoveContainer" containerID="838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41"
Jul 15 05:30:35.269312 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:30:35.269282    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41\": container with ID starting with 838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41 not found: ID does not exist" containerID="838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41"
Jul 15 05:30:35.269433 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.269321    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41"} err="failed to get container status \"838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41\": rpc error: code = NotFound desc = could not find container \"838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41\": container with ID starting with 838cfa5072db20d684928e8107200e33e1354b690d7c956b94a2ea4465ceeb41 not found: ID does not exist"
Jul 15 05:30:35.271301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.271268    2581 patch_prober.go:28] interesting pod/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 05:30:35.271395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.271328    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 05:30:35.281503 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.281480    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"]
Jul 15 05:30:35.287900 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:35.287875    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ff1de9c2-69c74b6fc9-7j2sj"]
Jul 15 05:30:36.388811 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:36.388775    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a3f6018-1c07-47a4-ab7c-4503ec12798d" path="/var/lib/kubelet/pods/2a3f6018-1c07-47a4-ab7c-4503ec12798d/volumes"
Jul 15 05:30:39.144388 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.144356    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:30:39.181315 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181240    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-data\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181315 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181297    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-envoy\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181502 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181319    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-credential-socket\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181502 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q9p8\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-kube-api-access-7q9p8\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181502 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-podinfo\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181502 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-socket\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181502 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181480    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53cf54f9-ca18-44f2-9272-989c471fe5bf-istiod-ca-cert\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181521    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-certs\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181554    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-token\") pod \"53cf54f9-ca18-44f2-9272-989c471fe5bf\" (UID: \"53cf54f9-ca18-44f2-9272-989c471fe5bf\") "
Jul 15 05:30:39.181773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181593    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:30:39.181938 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.181885    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.182081 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.182023    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:30:39.182255 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.182227    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:30:39.182608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.182587    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53cf54f9-ca18-44f2-9272-989c471fe5bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:30:39.183162 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.183140    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:30:39.184141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.184114    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:30:39.184277 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.184135    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:30:39.184342 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.184320    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-kube-api-access-7q9p8" (OuterVolumeSpecName: "kube-api-access-7q9p8") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "kube-api-access-7q9p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:30:39.185048 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.185024    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "53cf54f9-ca18-44f2-9272-989c471fe5bf" (UID: "53cf54f9-ca18-44f2-9272-989c471fe5bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:30:39.282395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282372    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.282395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282396    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.282590 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282409    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q9p8\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-kube-api-access-7q9p8\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.282590 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282421    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.282590 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282432    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.282590 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282443    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53cf54f9-ca18-44f2-9272-989c471fe5bf-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.282590 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282456    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53cf54f9-ca18-44f2-9272-989c471fe5bf-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.282590 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.282468    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53cf54f9-ca18-44f2-9272-989c471fe5bf-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:30:39.283184 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.283159    2581 generic.go:356] "Generic (PLEG): container finished" podID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerID="6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e" exitCode=0
Jul 15 05:30:39.283278 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.283226    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"
Jul 15 05:30:39.283278 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.283261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" event={"ID":"53cf54f9-ca18-44f2-9272-989c471fe5bf","Type":"ContainerDied","Data":"6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e"}
Jul 15 05:30:39.283346 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.283327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr" event={"ID":"53cf54f9-ca18-44f2-9272-989c471fe5bf","Type":"ContainerDied","Data":"188a9dcd685be70d403178008fde8304f02410d648bd7fa3693b6e810a180c21"}
Jul 15 05:30:39.283380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.283357    2581 scope.go:117] "RemoveContainer" containerID="6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e"
Jul 15 05:30:39.293655 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.293639    2581 scope.go:117] "RemoveContainer" containerID="e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0"
Jul 15 05:30:39.301015 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.300999    2581 scope.go:117] "RemoveContainer" containerID="6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e"
Jul 15 05:30:39.301271 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:30:39.301249    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e\": container with ID starting with 6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e not found: ID does not exist" containerID="6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e"
Jul 15 05:30:39.301316 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.301278    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e"} err="failed to get container status \"6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e\": rpc error: code = NotFound desc = could not find container \"6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e\": container with ID starting with 6a0014e7130b7f609d0e2281033192cb06346fcc85065a27dc343bbcecfd1a1e not found: ID does not exist"
Jul 15 05:30:39.301316 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.301294    2581 scope.go:117] "RemoveContainer" containerID="e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0"
Jul 15 05:30:39.301516 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:30:39.301500    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0\": container with ID starting with e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0 not found: ID does not exist" containerID="e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0"
Jul 15 05:30:39.301557 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.301521    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0"} err="failed to get container status \"e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0\": rpc error: code = NotFound desc = could not find container \"e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0\": container with ID starting with e20cbb017a5866fe658fd7ff736b5b5bbbd5468749dea527a79d89e022aac6a0 not found: ID does not exist"
Jul 15 05:30:39.306161 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.306128    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"]
Jul 15 05:30:39.310326 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.310308    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ff1de9c2-gw-openshift-default-58497d6dff-m7jzr"]
Jul 15 05:30:39.990454 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990424    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"]
Jul 15 05:30:39.990820 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990807    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy"
Jul 15 05:30:39.990873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990822    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy"
Jul 15 05:30:39.990873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990839    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy"
Jul 15 05:30:39.990873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990845    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy"
Jul 15 05:30:39.990873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990851    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3f6018-1c07-47a4-ab7c-4503ec12798d" containerName="maas-api"
Jul 15 05:30:39.990873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990857    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3f6018-1c07-47a4-ab7c-4503ec12798d" containerName="maas-api"
Jul 15 05:30:39.990873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990868    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy"
Jul 15 05:30:39.990873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990874    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy"
Jul 15 05:30:39.991099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990886    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy"
Jul 15 05:30:39.991099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990891    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy"
Jul 15 05:30:39.991099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990952    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy"
Jul 15 05:30:39.991099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990961    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy"
Jul 15 05:30:39.991099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990967    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3f6018-1c07-47a4-ab7c-4503ec12798d" containerName="maas-api"
Jul 15 05:30:39.991099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.990974    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" containerName="istio-proxy"
Jul 15 05:30:39.995670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.995637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:39.998509 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:39.998489    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bd94e112-openshift-default-dockercfg-xsm5b\""
Jul 15 05:30:40.008391 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.008363    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"]
Jul 15 05:30:40.088996 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.088969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istiod-ca-cert\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.088996 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b5dh\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-kube-api-access-8b5dh\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.089230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089026    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-data\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.089230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-credential-socket\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.089230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089102    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-certs\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.089230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089120    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-token\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.089230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-socket\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.089230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-podinfo\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.089542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.089279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-envoy\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.190795 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.190758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-socket\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.190836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-podinfo\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.190883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-envoy\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.190916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istiod-ca-cert\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.190942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b5dh\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-kube-api-access-8b5dh\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.190989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-data\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-credential-socket\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-certs\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-token\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-socket\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191752 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-data\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191752 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-certs\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191752 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191647    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-credential-socket\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.191915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.191846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istiod-ca-cert\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.193183 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.193161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-envoy\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.193509 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.193491    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-podinfo\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.199010 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.198986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-token\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.199176 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.199152    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b5dh\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-kube-api-access-8b5dh\") pod \"e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") " pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.309142 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.309123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:40.390726 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.390692    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53cf54f9-ca18-44f2-9272-989c471fe5bf" path="/var/lib/kubelet/pods/53cf54f9-ca18-44f2-9272-989c471fe5bf/volumes"
Jul 15 05:30:40.458716 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.458686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"]
Jul 15 05:30:40.461613 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:30:40.461589    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ac9cea_7bd1_4992_9731_b00e8c61163b.slice/crio-afc718fbee40c6e8d91875f3a7d389043e313db63e2968c7917b5c2c04a2bfe9 WatchSource:0}: Error finding container afc718fbee40c6e8d91875f3a7d389043e313db63e2968c7917b5c2c04a2bfe9: Status 404 returned error can't find the container with id afc718fbee40c6e8d91875f3a7d389043e313db63e2968c7917b5c2c04a2bfe9
Jul 15 05:30:40.463822 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.463791    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:30:40.463908 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.463851    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:30:40.463908 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:40.463880    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:30:41.297019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.296976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" event={"ID":"e4ac9cea-7bd1-4992-9731-b00e8c61163b","Type":"ContainerStarted","Data":"7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb"}
Jul 15 05:30:41.297403 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.297043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" event={"ID":"e4ac9cea-7bd1-4992-9731-b00e8c61163b","Type":"ContainerStarted","Data":"afc718fbee40c6e8d91875f3a7d389043e313db63e2968c7917b5c2c04a2bfe9"}
Jul 15 05:30:41.310145 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.310123    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:41.315276 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.315254    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:41.321973 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.321935    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" podStartSLOduration=2.321919527 podStartE2EDuration="2.321919527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:30:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:30:41.318781417 +0000 UTC m=+1919.486378091" watchObservedRunningTime="2026-07-15 05:30:41.321919527 +0000 UTC m=+1919.489516199"
Jul 15 05:30:41.353069 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.353047    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"]
Jul 15 05:30:41.353512 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.353499    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="126fcf13-084a-438a-835c-e3875ee9eef7" containerName="istio-proxy"
Jul 15 05:30:41.356653 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.356635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.359583 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.359561    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bd94e112-serving-cert\""
Jul 15 05:30:41.366123 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.366101    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"]
Jul 15 05:30:41.403757 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.403735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b67096-12a0-479b-8e93-92c7b5aec3b8-openshift-service-ca\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.403882 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.403858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.403967 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.403948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdktn\" (UniqueName: \"kubernetes.io/projected/19b67096-12a0-479b-8e93-92c7b5aec3b8-kube-api-access-qdktn\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.504906 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.504873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdktn\" (UniqueName: \"kubernetes.io/projected/19b67096-12a0-479b-8e93-92c7b5aec3b8-kube-api-access-qdktn\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.505072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.504931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b67096-12a0-479b-8e93-92c7b5aec3b8-openshift-service-ca\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.505072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.505020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.505155 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:30:41.505140    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-serving-cert: secret "maas-api-e2e-ait-bd94e112-serving-cert" not found
Jul 15 05:30:41.505249 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:30:41.505237    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls podName:19b67096-12a0-479b-8e93-92c7b5aec3b8 nodeName:}" failed. No retries permitted until 2026-07-15 05:30:42.005186003 +0000 UTC m=+1920.172782654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls") pod "maas-api-e2e-ait-bd94e112-57574b49c6-rncfj" (UID: "19b67096-12a0-479b-8e93-92c7b5aec3b8") : secret "maas-api-e2e-ait-bd94e112-serving-cert" not found
Jul 15 05:30:41.505610 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.505593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b67096-12a0-479b-8e93-92c7b5aec3b8-openshift-service-ca\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:41.513494 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:41.513468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdktn\" (UniqueName: \"kubernetes.io/projected/19b67096-12a0-479b-8e93-92c7b5aec3b8-kube-api-access-qdktn\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:42.009613 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:42.009578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:42.012027 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:42.011996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls\") pod \"maas-api-e2e-ait-bd94e112-57574b49c6-rncfj\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:42.271954 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:42.271877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:42.302954 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:42.302925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:42.303836 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:42.303818    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:42.435995 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:42.435901    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"]
Jul 15 05:30:42.439347 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:30:42.439325    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b67096_12a0_479b_8e93_92c7b5aec3b8.slice/crio-b664e6f15c691d4443c38e0a3e74e35b09d64330d20e5a76f4873d37048326a0 WatchSource:0}: Error finding container b664e6f15c691d4443c38e0a3e74e35b09d64330d20e5a76f4873d37048326a0: Status 404 returned error can't find the container with id b664e6f15c691d4443c38e0a3e74e35b09d64330d20e5a76f4873d37048326a0
Jul 15 05:30:43.311340 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:43.311304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj" event={"ID":"19b67096-12a0-479b-8e93-92c7b5aec3b8","Type":"ContainerStarted","Data":"5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966"}
Jul 15 05:30:43.311340 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:43.311355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj" event={"ID":"19b67096-12a0-479b-8e93-92c7b5aec3b8","Type":"ContainerStarted","Data":"b664e6f15c691d4443c38e0a3e74e35b09d64330d20e5a76f4873d37048326a0"}
Jul 15 05:30:43.330613 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:43.330564    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj" podStartSLOduration=1.893284387 podStartE2EDuration="2.330549059s" totalImagesPullingTime="437.264672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:30:43.328637447 +0000 UTC m=+1921.496234136" watchObservedRunningTime="2026-07-15 05:30:43.330549059 +0000 UTC m=+1921.498145732"
Jul 15 05:30:44.316825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:44.316797    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:50.327801 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:50.327764    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:30:57.304488 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.304402    2581 patch_prober.go:28] interesting pod/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 15 05:30:57.304488 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.304458    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 15 05:30:57.382492 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.382450    2581 generic.go:356] "Generic (PLEG): container finished" podID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerID="7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb" exitCode=0
Jul 15 05:30:57.382671 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.382519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" event={"ID":"e4ac9cea-7bd1-4992-9731-b00e8c61163b","Type":"ContainerDied","Data":"7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb"}
Jul 15 05:30:57.382920 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.382906    2581 scope.go:117] "RemoveContainer" containerID="7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb"
Jul 15 05:30:57.383982 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.383946    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:30:57.384081 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.384018    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:30:57.384081 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:57.384047    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:30:58.400531 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:58.399585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" event={"ID":"e4ac9cea-7bd1-4992-9731-b00e8c61163b","Type":"ContainerStarted","Data":"d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657"}
Jul 15 05:30:59.309712 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:59.309681    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:30:59.309900 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:59.309878    2581 patch_prober.go:28] interesting pod/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 15 05:30:59.309949 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:30:59.309926    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 15 05:31:00.310014 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:00.309980    2581 patch_prober.go:28] interesting pod/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 15 05:31:00.310427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:00.310040    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 15 05:31:01.310302 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:01.310269    2581 patch_prober.go:28] interesting pod/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 15 05:31:01.310692 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:01.310324    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 15 05:31:02.313572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:02.313544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:31:02.313951 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:02.313803    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:31:02.314584 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:02.314567    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:31:47.298618 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.298567    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"]
Jul 15 05:31:47.299283 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.298831    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj" podUID="19b67096-12a0-479b-8e93-92c7b5aec3b8" containerName="maas-api" containerID="cri-o://5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966" gracePeriod=30
Jul 15 05:31:47.553454 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.553400    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:31:47.621776 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.621750    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls\") pod \"19b67096-12a0-479b-8e93-92c7b5aec3b8\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") "
Jul 15 05:31:47.621933 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.621838    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b67096-12a0-479b-8e93-92c7b5aec3b8-openshift-service-ca\") pod \"19b67096-12a0-479b-8e93-92c7b5aec3b8\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") "
Jul 15 05:31:47.621975 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.621937    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdktn\" (UniqueName: \"kubernetes.io/projected/19b67096-12a0-479b-8e93-92c7b5aec3b8-kube-api-access-qdktn\") pod \"19b67096-12a0-479b-8e93-92c7b5aec3b8\" (UID: \"19b67096-12a0-479b-8e93-92c7b5aec3b8\") "
Jul 15 05:31:47.622141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.622124    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19b67096-12a0-479b-8e93-92c7b5aec3b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19b67096-12a0-479b-8e93-92c7b5aec3b8" (UID: "19b67096-12a0-479b-8e93-92c7b5aec3b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:31:47.622268 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.622249    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b67096-12a0-479b-8e93-92c7b5aec3b8-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:31:47.623927 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.623901    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b67096-12a0-479b-8e93-92c7b5aec3b8-kube-api-access-qdktn" (OuterVolumeSpecName: "kube-api-access-qdktn") pod "19b67096-12a0-479b-8e93-92c7b5aec3b8" (UID: "19b67096-12a0-479b-8e93-92c7b5aec3b8"). InnerVolumeSpecName "kube-api-access-qdktn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:31:47.624020 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.623963    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19b67096-12a0-479b-8e93-92c7b5aec3b8" (UID: "19b67096-12a0-479b-8e93-92c7b5aec3b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:31:47.643340 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.643312    2581 generic.go:356] "Generic (PLEG): container finished" podID="19b67096-12a0-479b-8e93-92c7b5aec3b8" containerID="5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966" exitCode=0
Jul 15 05:31:47.643472 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.643373    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"
Jul 15 05:31:47.643472 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.643393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj" event={"ID":"19b67096-12a0-479b-8e93-92c7b5aec3b8","Type":"ContainerDied","Data":"5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966"}
Jul 15 05:31:47.643472 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.643458    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj" event={"ID":"19b67096-12a0-479b-8e93-92c7b5aec3b8","Type":"ContainerDied","Data":"b664e6f15c691d4443c38e0a3e74e35b09d64330d20e5a76f4873d37048326a0"}
Jul 15 05:31:47.643621 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.643485    2581 scope.go:117] "RemoveContainer" containerID="5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966"
Jul 15 05:31:47.652646 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.652618    2581 scope.go:117] "RemoveContainer" containerID="5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966"
Jul 15 05:31:47.652919 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:31:47.652900    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966\": container with ID starting with 5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966 not found: ID does not exist" containerID="5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966"
Jul 15 05:31:47.652983 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.652931    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966"} err="failed to get container status \"5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966\": rpc error: code = NotFound desc = could not find container \"5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966\": container with ID starting with 5120698ec4a20f38de9f3084d0d3a19941f577439f7f441f91f124e3bc1b8966 not found: ID does not exist"
Jul 15 05:31:47.666317 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.666297    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"]
Jul 15 05:31:47.671487 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.671468    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd94e112-57574b49c6-rncfj"]
Jul 15 05:31:47.723491 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.723462    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdktn\" (UniqueName: \"kubernetes.io/projected/19b67096-12a0-479b-8e93-92c7b5aec3b8-kube-api-access-qdktn\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:31:47.723491 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:47.723487    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b67096-12a0-479b-8e93-92c7b5aec3b8-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:31:48.389144 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:31:48.389113    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19b67096-12a0-479b-8e93-92c7b5aec3b8" path="/var/lib/kubelet/pods/19b67096-12a0-479b-8e93-92c7b5aec3b8/volumes"
Jul 15 05:32:02.704612 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:02.704572    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"]
Jul 15 05:32:02.705093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:02.704778    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy" containerID="cri-o://d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657" gracePeriod=30
Jul 15 05:32:03.103251 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.103216    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"]
Jul 15 05:32:03.103714 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.103696    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19b67096-12a0-479b-8e93-92c7b5aec3b8" containerName="maas-api"
Jul 15 05:32:03.103762 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.103720    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="19b67096-12a0-479b-8e93-92c7b5aec3b8" containerName="maas-api"
Jul 15 05:32:03.103855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.103842    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="19b67096-12a0-479b-8e93-92c7b5aec3b8" containerName="maas-api"
Jul 15 05:32:03.107541 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.107522    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.111595 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.111572    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7fde7053-openshift-default-dockercfg-lhgwj\""
Jul 15 05:32:03.128616 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.128594    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"]
Jul 15 05:32:03.162464 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-token\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-envoy\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-socket\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162716 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-credential-socket\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162716 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-certs\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162716 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-data\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162821 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istiod-ca-cert\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162821 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-podinfo\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.162821 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.162788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6prl\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-kube-api-access-m6prl\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264197 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istiod-ca-cert\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264197 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-podinfo\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6prl\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-kube-api-access-m6prl\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-token\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-envoy\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-socket\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-credential-socket\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264691 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-certs\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264691 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-data\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264799 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-credential-socket\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-socket\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-data\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.264948 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-certs\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.265013 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.264988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istiod-ca-cert\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.267170 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.267144    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-envoy\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.267313 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.267297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-podinfo\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.274619 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.274576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6prl\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-kube-api-access-m6prl\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.275599 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.275576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-token\") pod \"e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") " pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.418084 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.418010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:03.563501 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.563474    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"]
Jul 15 05:32:03.565498 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:32:03.565473    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2c66bf_4f6a_40f2_8c64_066a6b165937.slice/crio-5bf4b766d5f540e9f8a6742ab91d4108aa861476612fa7662e127327d860de27 WatchSource:0}: Error finding container 5bf4b766d5f540e9f8a6742ab91d4108aa861476612fa7662e127327d860de27: Status 404 returned error can't find the container with id 5bf4b766d5f540e9f8a6742ab91d4108aa861476612fa7662e127327d860de27
Jul 15 05:32:03.567778 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.567743    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:32:03.567840 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.567824    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:32:03.567878 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.567865    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:32:03.729334 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.729236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" event={"ID":"cd2c66bf-4f6a-40f2-8c64-066a6b165937","Type":"ContainerStarted","Data":"5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1"}
Jul 15 05:32:03.729334 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.729294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" event={"ID":"cd2c66bf-4f6a-40f2-8c64-066a6b165937","Type":"ContainerStarted","Data":"5bf4b766d5f540e9f8a6742ab91d4108aa861476612fa7662e127327d860de27"}
Jul 15 05:32:03.757385 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:03.757338    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" podStartSLOduration=0.757321048 podStartE2EDuration="757.321048ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:32:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:32:03.754878367 +0000 UTC m=+2001.922475061" watchObservedRunningTime="2026-07-15 05:32:03.757321048 +0000 UTC m=+2001.924917719"
Jul 15 05:32:04.419131 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.419098    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:04.424083 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.424057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:04.461816 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.461781    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"]
Jul 15 05:32:04.471675 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.471066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.474661 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.474635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7fde7053-serving-cert\""
Jul 15 05:32:04.486087 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.486057    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"]
Jul 15 05:32:04.578969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.578943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqjcc\" (UniqueName: \"kubernetes.io/projected/69a7e48c-24bc-49e6-965d-44293075ab7f-kube-api-access-bqjcc\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.579120 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.579027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7e48c-24bc-49e6-965d-44293075ab7f-maas-api-tls\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.579120 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.579055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7e48c-24bc-49e6-965d-44293075ab7f-openshift-service-ca\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.679906 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.679805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqjcc\" (UniqueName: \"kubernetes.io/projected/69a7e48c-24bc-49e6-965d-44293075ab7f-kube-api-access-bqjcc\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.680093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.679964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7e48c-24bc-49e6-965d-44293075ab7f-maas-api-tls\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.680093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.680007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7e48c-24bc-49e6-965d-44293075ab7f-openshift-service-ca\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.680895 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.680871    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7e48c-24bc-49e6-965d-44293075ab7f-openshift-service-ca\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.682477 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.682458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7e48c-24bc-49e6-965d-44293075ab7f-maas-api-tls\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.690069 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.690040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqjcc\" (UniqueName: \"kubernetes.io/projected/69a7e48c-24bc-49e6-965d-44293075ab7f-kube-api-access-bqjcc\") pod \"maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.735023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.734994    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:04.735956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.735940    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:04.785319 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.785293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:04.937325 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:04.937303    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"]
Jul 15 05:32:04.938846 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:32:04.938825    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a7e48c_24bc_49e6_965d_44293075ab7f.slice/crio-a80932b0304cec96b6d0f4bdbe140b7dbf5c39609704b017e77398d77cb13dbc WatchSource:0}: Error finding container a80932b0304cec96b6d0f4bdbe140b7dbf5c39609704b017e77398d77cb13dbc: Status 404 returned error can't find the container with id a80932b0304cec96b6d0f4bdbe140b7dbf5c39609704b017e77398d77cb13dbc
Jul 15 05:32:05.742973 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:05.742941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp" event={"ID":"69a7e48c-24bc-49e6-965d-44293075ab7f","Type":"ContainerStarted","Data":"a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934"}
Jul 15 05:32:05.743405 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:05.742995    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp" event={"ID":"69a7e48c-24bc-49e6-965d-44293075ab7f","Type":"ContainerStarted","Data":"a80932b0304cec96b6d0f4bdbe140b7dbf5c39609704b017e77398d77cb13dbc"}
Jul 15 05:32:05.743405 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:05.743195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:05.766976 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:05.766923    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp" podStartSLOduration=1.117446016 podStartE2EDuration="1.766909743s" totalImagesPullingTime="649.463727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:32:05.764916713 +0000 UTC m=+2003.932513385" watchObservedRunningTime="2026-07-15 05:32:05.766909743 +0000 UTC m=+2003.934506414"
Jul 15 05:32:07.961333 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:07.961307    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:32:08.112453 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112423    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-token\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112495    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-data\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112561    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-certs\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112614 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112592    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-podinfo\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112623    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-envoy\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-socket\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112705    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b5dh\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-kube-api-access-8b5dh\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112736    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istiod-ca-cert\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.112783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112770    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-credential-socket\") pod \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\" (UID: \"e4ac9cea-7bd1-4992-9731-b00e8c61163b\") "
Jul 15 05:32:08.113041 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.112835    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:32:08.113095 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.113053    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:32:08.113168 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.113148    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.113168 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.113155    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:32:08.113313 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.113171    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.113383 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.113364    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:32:08.113514 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.113487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:32:08.114708 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.114689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:32:08.114821 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.114721    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:32:08.115090 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.115073    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-kube-api-access-8b5dh" (OuterVolumeSpecName: "kube-api-access-8b5dh") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "kube-api-access-8b5dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:32:08.115133 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.115089    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4ac9cea-7bd1-4992-9731-b00e8c61163b" (UID: "e4ac9cea-7bd1-4992-9731-b00e8c61163b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:32:08.213587 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.213567    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.213587 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.213588    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.213715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.213596    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.213715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.213604    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.213715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.213614    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4ac9cea-7bd1-4992-9731-b00e8c61163b-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.213715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.213622    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b5dh\" (UniqueName: \"kubernetes.io/projected/e4ac9cea-7bd1-4992-9731-b00e8c61163b-kube-api-access-8b5dh\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.213715 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.213631    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4ac9cea-7bd1-4992-9731-b00e8c61163b-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:32:08.768445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.768324    2581 generic.go:356] "Generic (PLEG): container finished" podID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerID="d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657" exitCode=0
Jul 15 05:32:08.768445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.768363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" event={"ID":"e4ac9cea-7bd1-4992-9731-b00e8c61163b","Type":"ContainerDied","Data":"d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657"}
Jul 15 05:32:08.768445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.768403    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"
Jul 15 05:32:08.768720 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.768700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l" event={"ID":"e4ac9cea-7bd1-4992-9731-b00e8c61163b","Type":"ContainerDied","Data":"afc718fbee40c6e8d91875f3a7d389043e313db63e2968c7917b5c2c04a2bfe9"}
Jul 15 05:32:08.768781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.768731    2581 scope.go:117] "RemoveContainer" containerID="d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657"
Jul 15 05:32:08.778591 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.778571    2581 scope.go:117] "RemoveContainer" containerID="7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb"
Jul 15 05:32:08.787111 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.787095    2581 scope.go:117] "RemoveContainer" containerID="d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657"
Jul 15 05:32:08.787372 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:32:08.787353    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657\": container with ID starting with d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657 not found: ID does not exist" containerID="d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657"
Jul 15 05:32:08.787422 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.787379    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657"} err="failed to get container status \"d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657\": rpc error: code = NotFound desc = could not find container \"d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657\": container with ID starting with d8ebb95daaadd2023dccb47853a79eaafc126094b96d590fddd0c3b00d337657 not found: ID does not exist"
Jul 15 05:32:08.787422 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.787396    2581 scope.go:117] "RemoveContainer" containerID="7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb"
Jul 15 05:32:08.787686 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:32:08.787663    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb\": container with ID starting with 7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb not found: ID does not exist" containerID="7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb"
Jul 15 05:32:08.787754 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.787696    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb"} err="failed to get container status \"7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb\": rpc error: code = NotFound desc = could not find container \"7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb\": container with ID starting with 7309a454ba9ca1cb118cb75e4cde98a89cbaa2b96ba437b3ea6e40641e59f0fb not found: ID does not exist"
Jul 15 05:32:08.794779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.794756    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"]
Jul 15 05:32:08.807667 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:08.807645    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bd94e112-openshift-default-5f8b44fb7f-6672l"]
Jul 15 05:32:10.388467 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:10.388438    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" path="/var/lib/kubelet/pods/e4ac9cea-7bd1-4992-9731-b00e8c61163b/volumes"
Jul 15 05:32:11.753544 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:11.753515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:32:19.924320 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:19.924027    2581 patch_prober.go:28] interesting pod/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:46338->10.134.0.61:15021: read: connection reset by peer" start-of-body=
Jul 15 05:32:19.924320 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:19.924083    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:46338->10.134.0.61:15021: read: connection reset by peer"
Jul 15 05:32:20.841010 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:20.840980    2581 generic.go:356] "Generic (PLEG): container finished" podID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerID="5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1" exitCode=0
Jul 15 05:32:20.841179 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:20.841032    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" event={"ID":"cd2c66bf-4f6a-40f2-8c64-066a6b165937","Type":"ContainerDied","Data":"5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1"}
Jul 15 05:32:20.841449 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:20.841435    2581 scope.go:117] "RemoveContainer" containerID="5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1"
Jul 15 05:32:20.842452 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:20.842423    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:32:20.842517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:20.842490    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:32:20.842553 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:20.842523    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:32:21.849608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:21.849577    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" event={"ID":"cd2c66bf-4f6a-40f2-8c64-066a6b165937","Type":"ContainerStarted","Data":"8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842"}
Jul 15 05:32:22.419113 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:22.419072    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:22.419342 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:22.419320    2581 patch_prober.go:28] interesting pod/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 15 05:32:22.419404 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:22.419368    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 15 05:32:23.418740 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:23.418707    2581 patch_prober.go:28] interesting pod/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 15 05:32:23.419094 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:23.418775    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 15 05:32:24.418578 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:24.418543    2581 patch_prober.go:28] interesting pod/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 15 05:32:24.418746 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:24.418596    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 15 05:32:25.418900 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:25.418863    2581 patch_prober.go:28] interesting pod/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 15 05:32:25.419330 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:25.418925    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 15 05:32:26.421793 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:26.421766    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:26.422174 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:26.422024    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:32:26.422740 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:32:26.422720    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:33:19.190681 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.190590    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"]
Jul 15 05:33:19.191180 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.190837    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp" podUID="69a7e48c-24bc-49e6-965d-44293075ab7f" containerName="maas-api" containerID="cri-o://a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934" gracePeriod=30
Jul 15 05:33:19.454396 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.454339    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:33:19.629075 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.629045    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7e48c-24bc-49e6-965d-44293075ab7f-maas-api-tls\") pod \"69a7e48c-24bc-49e6-965d-44293075ab7f\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") "
Jul 15 05:33:19.629272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.629135    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqjcc\" (UniqueName: \"kubernetes.io/projected/69a7e48c-24bc-49e6-965d-44293075ab7f-kube-api-access-bqjcc\") pod \"69a7e48c-24bc-49e6-965d-44293075ab7f\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") "
Jul 15 05:33:19.629272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.629153    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7e48c-24bc-49e6-965d-44293075ab7f-openshift-service-ca\") pod \"69a7e48c-24bc-49e6-965d-44293075ab7f\" (UID: \"69a7e48c-24bc-49e6-965d-44293075ab7f\") "
Jul 15 05:33:19.629564 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.629541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a7e48c-24bc-49e6-965d-44293075ab7f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69a7e48c-24bc-49e6-965d-44293075ab7f" (UID: "69a7e48c-24bc-49e6-965d-44293075ab7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:33:19.630963 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.630939    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69a7e48c-24bc-49e6-965d-44293075ab7f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69a7e48c-24bc-49e6-965d-44293075ab7f" (UID: "69a7e48c-24bc-49e6-965d-44293075ab7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:33:19.631063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.631048    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a7e48c-24bc-49e6-965d-44293075ab7f-kube-api-access-bqjcc" (OuterVolumeSpecName: "kube-api-access-bqjcc") pod "69a7e48c-24bc-49e6-965d-44293075ab7f" (UID: "69a7e48c-24bc-49e6-965d-44293075ab7f"). InnerVolumeSpecName "kube-api-access-bqjcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:33:19.729970 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.729898    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69a7e48c-24bc-49e6-965d-44293075ab7f-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:19.729970 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.729923    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqjcc\" (UniqueName: \"kubernetes.io/projected/69a7e48c-24bc-49e6-965d-44293075ab7f-kube-api-access-bqjcc\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:19.729970 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:19.729937    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69a7e48c-24bc-49e6-965d-44293075ab7f-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:20.159005 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.158972    2581 generic.go:356] "Generic (PLEG): container finished" podID="69a7e48c-24bc-49e6-965d-44293075ab7f" containerID="a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934" exitCode=0
Jul 15 05:33:20.159173 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.159015    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"
Jul 15 05:33:20.159173 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.159056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp" event={"ID":"69a7e48c-24bc-49e6-965d-44293075ab7f","Type":"ContainerDied","Data":"a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934"}
Jul 15 05:33:20.159173 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.159118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp" event={"ID":"69a7e48c-24bc-49e6-965d-44293075ab7f","Type":"ContainerDied","Data":"a80932b0304cec96b6d0f4bdbe140b7dbf5c39609704b017e77398d77cb13dbc"}
Jul 15 05:33:20.159173 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.159140    2581 scope.go:117] "RemoveContainer" containerID="a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934"
Jul 15 05:33:20.168957 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.168940    2581 scope.go:117] "RemoveContainer" containerID="a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934"
Jul 15 05:33:20.169298 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:33:20.169185    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934\": container with ID starting with a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934 not found: ID does not exist" containerID="a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934"
Jul 15 05:33:20.169298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.169233    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934"} err="failed to get container status \"a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934\": rpc error: code = NotFound desc = could not find container \"a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934\": container with ID starting with a2ba0e1f92920bf6ee9bf553f931a4df18d4e91ac8c4aff7d9ff8f1eeb931934 not found: ID does not exist"
Jul 15 05:33:20.181935 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.181911    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"]
Jul 15 05:33:20.184868 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.184850    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fde7053-57f6c79b8b-nmrgp"]
Jul 15 05:33:20.388161 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:20.388131    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69a7e48c-24bc-49e6-965d-44293075ab7f" path="/var/lib/kubelet/pods/69a7e48c-24bc-49e6-965d-44293075ab7f/volumes"
Jul 15 05:33:34.283731 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:34.283693    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"]
Jul 15 05:33:34.284308 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:34.283931    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy" containerID="cri-o://8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842" gracePeriod=30
Jul 15 05:33:39.538819 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.538791    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:33:39.602536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602509    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istiod-ca-cert\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602688 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602558    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-data\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602688 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602578    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6prl\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-kube-api-access-m6prl\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602688 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602618    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-socket\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602689    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-credential-socket\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-podinfo\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-envoy\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-token\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.602855 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602845    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-certs\") pod \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\" (UID: \"cd2c66bf-4f6a-40f2-8c64-066a6b165937\") "
Jul 15 05:33:39.603098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.602985    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:33:39.603244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.603189    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.603325 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.603269    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:33:39.603650 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.603523    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:33:39.603872 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.603838    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:33:39.604134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.604108    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:33:39.605506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.605452    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:33:39.605615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.605530    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:33:39.605615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.605553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:33:39.605781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.605763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-kube-api-access-m6prl" (OuterVolumeSpecName: "kube-api-access-m6prl") pod "cd2c66bf-4f6a-40f2-8c64-066a6b165937" (UID: "cd2c66bf-4f6a-40f2-8c64-066a6b165937"). InnerVolumeSpecName "kube-api-access-m6prl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:33:39.703636 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703610    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.703636 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703635    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.703774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703644    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.703774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703652    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.703774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703660    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.703774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703667    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.703774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703674    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c66bf-4f6a-40f2-8c64-066a6b165937-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:39.703774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:39.703682    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6prl\" (UniqueName: \"kubernetes.io/projected/cd2c66bf-4f6a-40f2-8c64-066a6b165937-kube-api-access-m6prl\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:33:40.266147 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.266115    2581 generic.go:356] "Generic (PLEG): container finished" podID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerID="8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842" exitCode=0
Jul 15 05:33:40.266368 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.266170    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"
Jul 15 05:33:40.266368 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.266226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" event={"ID":"cd2c66bf-4f6a-40f2-8c64-066a6b165937","Type":"ContainerDied","Data":"8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842"}
Jul 15 05:33:40.266368 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.266293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx" event={"ID":"cd2c66bf-4f6a-40f2-8c64-066a6b165937","Type":"ContainerDied","Data":"5bf4b766d5f540e9f8a6742ab91d4108aa861476612fa7662e127327d860de27"}
Jul 15 05:33:40.266368 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.266325    2581 scope.go:117] "RemoveContainer" containerID="8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842"
Jul 15 05:33:40.276447 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.276431    2581 scope.go:117] "RemoveContainer" containerID="5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1"
Jul 15 05:33:40.284427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.284390    2581 scope.go:117] "RemoveContainer" containerID="8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842"
Jul 15 05:33:40.284661 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:33:40.284643    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842\": container with ID starting with 8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842 not found: ID does not exist" containerID="8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842"
Jul 15 05:33:40.284742 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.284673    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842"} err="failed to get container status \"8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842\": rpc error: code = NotFound desc = could not find container \"8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842\": container with ID starting with 8fe80daee546107fa02db44be88f6308739da0d5d25c6b16f65766cb08c4a842 not found: ID does not exist"
Jul 15 05:33:40.284742 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.284697    2581 scope.go:117] "RemoveContainer" containerID="5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1"
Jul 15 05:33:40.285029 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:33:40.285007    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1\": container with ID starting with 5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1 not found: ID does not exist" containerID="5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1"
Jul 15 05:33:40.285122 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.285038    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1"} err="failed to get container status \"5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1\": rpc error: code = NotFound desc = could not find container \"5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1\": container with ID starting with 5a680f46b799e5e4b78a916e1a42916387ee84cfffc2844fbc412b943f8cabe1 not found: ID does not exist"
Jul 15 05:33:40.287581 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.287562    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"]
Jul 15 05:33:40.293177 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.293159    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7fde7053-openshift-default-69bc45f999-cccmx"]
Jul 15 05:33:40.389627 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:33:40.389601    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" path="/var/lib/kubelet/pods/cd2c66bf-4f6a-40f2-8c64-066a6b165937/volumes"
Jul 15 05:35:14.101468 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101437    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"]
Jul 15 05:35:14.101897 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101884    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy"
Jul 15 05:35:14.101897 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101897    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy"
Jul 15 05:35:14.101969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101925    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy"
Jul 15 05:35:14.101969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101931    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy"
Jul 15 05:35:14.101969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101938    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy"
Jul 15 05:35:14.101969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101944    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy"
Jul 15 05:35:14.101969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101955    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a7e48c-24bc-49e6-965d-44293075ab7f" containerName="maas-api"
Jul 15 05:35:14.101969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.101964    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a7e48c-24bc-49e6-965d-44293075ab7f" containerName="maas-api"
Jul 15 05:35:14.102146 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.102045    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy"
Jul 15 05:35:14.102146 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.102055    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ac9cea-7bd1-4992-9731-b00e8c61163b" containerName="istio-proxy"
Jul 15 05:35:14.102146 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.102063    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy"
Jul 15 05:35:14.102146 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.102069    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy"
Jul 15 05:35:14.102146 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.102074    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69a7e48c-24bc-49e6-965d-44293075ab7f" containerName="maas-api"
Jul 15 05:35:14.104465 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.104449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.107150 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.107132    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1fa2b35b-openshift-default-dockercfg-b8rrv\""
Jul 15 05:35:14.126865 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.126838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"]
Jul 15 05:35:14.184361 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184327    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-podinfo\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184499 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-envoy\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184499 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-certs\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184601 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-data\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184601 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istiod-ca-cert\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184601 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-credential-socket\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184750 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mqqf\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-kube-api-access-4mqqf\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184750 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-socket\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.184750 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.184670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-token\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.285837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-certs\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.285923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-data\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.286016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istiod-ca-cert\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.286051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-credential-socket\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.286095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mqqf\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-kube-api-access-4mqqf\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.286130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-socket\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.286159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-token\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.286284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-podinfo\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.286348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-envoy\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.287077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-certs\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.287346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-data\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.287933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istiod-ca-cert\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.288171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-credential-socket\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289230 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.288745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-socket\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.289999 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.289335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-envoy\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.293064 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.292029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-podinfo\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.296475 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.296447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mqqf\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-kube-api-access-4mqqf\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.300745 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.300723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-token\") pod \"e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") " pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.415409 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.415342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:14.557332 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.557302    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"]
Jul 15 05:35:14.558446 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:35:14.558422    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf86d30_4ccf_4562_9edc_4c9d9014d803.slice/crio-dd2b83b5169da1bb2aefa75df465519275f1b66e347a70f6aac7f863fe283239 WatchSource:0}: Error finding container dd2b83b5169da1bb2aefa75df465519275f1b66e347a70f6aac7f863fe283239: Status 404 returned error can't find the container with id dd2b83b5169da1bb2aefa75df465519275f1b66e347a70f6aac7f863fe283239
Jul 15 05:35:14.560560 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.560526    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:35:14.560608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.560592    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:35:14.560644 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.560618    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:35:14.733099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.732998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" event={"ID":"5bf86d30-4ccf-4562-9edc-4c9d9014d803","Type":"ContainerStarted","Data":"515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511"}
Jul 15 05:35:14.733099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.733066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" event={"ID":"5bf86d30-4ccf-4562-9edc-4c9d9014d803","Type":"ContainerStarted","Data":"dd2b83b5169da1bb2aefa75df465519275f1b66e347a70f6aac7f863fe283239"}
Jul 15 05:35:14.752498 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:14.752445    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podStartSLOduration=0.752430475 podStartE2EDuration="752.430475ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:35:14.749798081 +0000 UTC m=+2192.917394768" watchObservedRunningTime="2026-07-15 05:35:14.752430475 +0000 UTC m=+2192.920027217"
Jul 15 05:35:15.416497 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:15.416467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:15.418023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:15.417998    2581 patch_prober.go:28] interesting pod/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 15 05:35:15.418138 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:15.418044    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 15 05:35:16.420417 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.420383    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:16.743759 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.743675    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:16.744648 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.744626    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:16.919411 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.919376    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"]
Jul 15 05:35:16.920007 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.919980    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy"
Jul 15 05:35:16.920007 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.920006    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2c66bf-4f6a-40f2-8c64-066a6b165937" containerName="istio-proxy"
Jul 15 05:35:16.928117 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.928098    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:16.932427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.932402    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1fa2b35b-serving-cert\""
Jul 15 05:35:16.933236 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:16.933196    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"]
Jul 15 05:35:17.010130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.010052    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b78879-4f4a-460a-a70f-bdb9c052fe2e-openshift-service-ca\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.010130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.010105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbd6s\" (UniqueName: \"kubernetes.io/projected/47b78879-4f4a-460a-a70f-bdb9c052fe2e-kube-api-access-qbd6s\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.010302 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.010178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.111093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.111056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b78879-4f4a-460a-a70f-bdb9c052fe2e-openshift-service-ca\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.111093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.111097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbd6s\" (UniqueName: \"kubernetes.io/projected/47b78879-4f4a-460a-a70f-bdb9c052fe2e-kube-api-access-qbd6s\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.111317 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.111236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.111380 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:35:17.111361    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-serving-cert: secret "maas-api-e2e-mt-1fa2b35b-serving-cert" not found
Jul 15 05:35:17.111447 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:35:17.111434    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls podName:47b78879-4f4a-460a-a70f-bdb9c052fe2e nodeName:}" failed. No retries permitted until 2026-07-15 05:35:17.611413548 +0000 UTC m=+2195.779010204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls") pod "maas-api-e2e-mt-1fa2b35b-847d66685-7sclq" (UID: "47b78879-4f4a-460a-a70f-bdb9c052fe2e") : secret "maas-api-e2e-mt-1fa2b35b-serving-cert" not found
Jul 15 05:35:17.111798 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.111781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b78879-4f4a-460a-a70f-bdb9c052fe2e-openshift-service-ca\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.121781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.121748    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbd6s\" (UniqueName: \"kubernetes.io/projected/47b78879-4f4a-460a-a70f-bdb9c052fe2e-kube-api-access-qbd6s\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.616840 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.616804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.619113 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.619088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls\") pod \"maas-api-e2e-mt-1fa2b35b-847d66685-7sclq\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.841481 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.841453    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:17.995652 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.995624    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"]
Jul 15 05:35:17.997404 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:35:17.997373    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b78879_4f4a_460a_a70f_bdb9c052fe2e.slice/crio-8ce80e5b92a0ab543c3806239167e4746dfbe9391086cc9843c0541271391b98 WatchSource:0}: Error finding container 8ce80e5b92a0ab543c3806239167e4746dfbe9391086cc9843c0541271391b98: Status 404 returned error can't find the container with id 8ce80e5b92a0ab543c3806239167e4746dfbe9391086cc9843c0541271391b98
Jul 15 05:35:17.998854 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:17.998837    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:35:18.757098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:18.757002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq" event={"ID":"47b78879-4f4a-460a-a70f-bdb9c052fe2e","Type":"ContainerStarted","Data":"dbfc8527c3f68742d4fe508c66c88db3836cb29db43f4f177ba40e1e4ce47f73"}
Jul 15 05:35:18.757098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:18.757060    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq" event={"ID":"47b78879-4f4a-460a-a70f-bdb9c052fe2e","Type":"ContainerStarted","Data":"8ce80e5b92a0ab543c3806239167e4746dfbe9391086cc9843c0541271391b98"}
Jul 15 05:35:18.757508 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:18.757189    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:18.773805 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:18.773751    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq" podStartSLOduration=2.290404566 podStartE2EDuration="2.773733488s" totalImagesPullingTime="483.328922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:35:18.773071194 +0000 UTC m=+2196.940667868" watchObservedRunningTime="2026-07-15 05:35:18.773733488 +0000 UTC m=+2196.941330162"
Jul 15 05:35:24.767933 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:24.767897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:31.744760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.744723    2581 patch_prober.go:28] interesting pod/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 15 05:35:31.745156 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.744784    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 15 05:35:31.836638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.836605    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerID="515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511" exitCode=0
Jul 15 05:35:31.836785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.836681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" event={"ID":"5bf86d30-4ccf-4562-9edc-4c9d9014d803","Type":"ContainerDied","Data":"515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511"}
Jul 15 05:35:31.837065 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.837051    2581 scope.go:117] "RemoveContainer" containerID="515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511"
Jul 15 05:35:31.838067 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.838033    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:35:31.838165 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.838120    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:35:31.838226 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:31.838168    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:35:32.844977 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:32.844944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" event={"ID":"5bf86d30-4ccf-4562-9edc-4c9d9014d803","Type":"ContainerStarted","Data":"d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8"}
Jul 15 05:35:33.415499 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:33.415467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:33.415674 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:33.415649    2581 patch_prober.go:28] interesting pod/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 15 05:35:33.415740 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:33.415717    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 15 05:35:34.415973 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:34.415940    2581 patch_prober.go:28] interesting pod/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 15 05:35:34.416363 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:34.415996    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 15 05:35:35.416438 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:35.416400    2581 patch_prober.go:28] interesting pod/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 15 05:35:35.416813 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:35.416456    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 15 05:35:36.415792 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:36.415750    2581 patch_prober.go:28] interesting pod/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 15 05:35:36.415971 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:36.415830    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 15 05:35:37.419294 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:37.419266    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:37.419671 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:37.419642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:37.420323 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:37.420307    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:38.729836 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:38.729803    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"]
Jul 15 05:35:38.730338 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:38.730070    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq" podUID="47b78879-4f4a-460a-a70f-bdb9c052fe2e" containerName="maas-api" containerID="cri-o://dbfc8527c3f68742d4fe508c66c88db3836cb29db43f4f177ba40e1e4ce47f73" gracePeriod=30
Jul 15 05:35:38.878718 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:38.878682    2581 generic.go:356] "Generic (PLEG): container finished" podID="47b78879-4f4a-460a-a70f-bdb9c052fe2e" containerID="dbfc8527c3f68742d4fe508c66c88db3836cb29db43f4f177ba40e1e4ce47f73" exitCode=0
Jul 15 05:35:38.878885 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:38.878756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq" event={"ID":"47b78879-4f4a-460a-a70f-bdb9c052fe2e","Type":"ContainerDied","Data":"dbfc8527c3f68742d4fe508c66c88db3836cb29db43f4f177ba40e1e4ce47f73"}
Jul 15 05:35:38.996711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:38.996683    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:39.146186 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.146160    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbd6s\" (UniqueName: \"kubernetes.io/projected/47b78879-4f4a-460a-a70f-bdb9c052fe2e-kube-api-access-qbd6s\") pod \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") "
Jul 15 05:35:39.146384 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.146253    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls\") pod \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") "
Jul 15 05:35:39.146384 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.146281    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b78879-4f4a-460a-a70f-bdb9c052fe2e-openshift-service-ca\") pod \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\" (UID: \"47b78879-4f4a-460a-a70f-bdb9c052fe2e\") "
Jul 15 05:35:39.146676 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.146652    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47b78879-4f4a-460a-a70f-bdb9c052fe2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "47b78879-4f4a-460a-a70f-bdb9c052fe2e" (UID: "47b78879-4f4a-460a-a70f-bdb9c052fe2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:35:39.148174 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.148150    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "47b78879-4f4a-460a-a70f-bdb9c052fe2e" (UID: "47b78879-4f4a-460a-a70f-bdb9c052fe2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:35:39.148278 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.148174    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b78879-4f4a-460a-a70f-bdb9c052fe2e-kube-api-access-qbd6s" (OuterVolumeSpecName: "kube-api-access-qbd6s") pod "47b78879-4f4a-460a-a70f-bdb9c052fe2e" (UID: "47b78879-4f4a-460a-a70f-bdb9c052fe2e"). InnerVolumeSpecName "kube-api-access-qbd6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:35:39.247667 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.247602    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbd6s\" (UniqueName: \"kubernetes.io/projected/47b78879-4f4a-460a-a70f-bdb9c052fe2e-kube-api-access-qbd6s\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:39.247667 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.247628    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b78879-4f4a-460a-a70f-bdb9c052fe2e-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:39.247667 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.247637    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b78879-4f4a-460a-a70f-bdb9c052fe2e-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:39.886022 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.885986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq" event={"ID":"47b78879-4f4a-460a-a70f-bdb9c052fe2e","Type":"ContainerDied","Data":"8ce80e5b92a0ab543c3806239167e4746dfbe9391086cc9843c0541271391b98"}
Jul 15 05:35:39.886437 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.886051    2581 scope.go:117] "RemoveContainer" containerID="dbfc8527c3f68742d4fe508c66c88db3836cb29db43f4f177ba40e1e4ce47f73"
Jul 15 05:35:39.886437 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.885999    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"
Jul 15 05:35:39.907283 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.907260    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"]
Jul 15 05:35:39.911032 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:39.911012    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fa2b35b-847d66685-7sclq"]
Jul 15 05:35:40.388513 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:40.388482    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47b78879-4f4a-460a-a70f-bdb9c052fe2e" path="/var/lib/kubelet/pods/47b78879-4f4a-460a-a70f-bdb9c052fe2e/volumes"
Jul 15 05:35:54.131952 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:54.131919    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"]
Jul 15 05:35:54.132371 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:54.132188    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy" containerID="cri-o://d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8" gracePeriod=30
Jul 15 05:35:59.383108 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.383084    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:59.413341 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413277    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-data\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413341 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413337    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-certs\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413522 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413382    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-token\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413522 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413413    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mqqf\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-kube-api-access-4mqqf\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413522 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413442    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-socket\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413522 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413471    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istiod-ca-cert\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413522 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413505    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-podinfo\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413786 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413565    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:35:59.413786 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413594    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-envoy\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413786 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413660    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-credential-socket\") pod \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\" (UID: \"5bf86d30-4ccf-4562-9edc-4c9d9014d803\") "
Jul 15 05:35:59.413954 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.413858    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:35:59.414063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.414043    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.414139 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.414078    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.414215 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.414154    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:35:59.414283 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.414249    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-data" (OuterVolumeSpecName: "istio-data") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:35:59.415609 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.415213    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:35:59.416650 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.416625    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-kube-api-access-4mqqf" (OuterVolumeSpecName: "kube-api-access-4mqqf") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "kube-api-access-4mqqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:35:59.417586 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.417550    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-token" (OuterVolumeSpecName: "istio-token") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:35:59.424482 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.424451    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:35:59.426987 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.426957    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5bf86d30-4ccf-4562-9edc-4c9d9014d803" (UID: "5bf86d30-4ccf-4562-9edc-4c9d9014d803"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:35:59.514938 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.514917    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.514938 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.514937    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.515063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.514964    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mqqf\" (UniqueName: \"kubernetes.io/projected/5bf86d30-4ccf-4562-9edc-4c9d9014d803-kube-api-access-4mqqf\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.515063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.514974    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.515063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.514982    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.515063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.514990    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.515063 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.514997    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf86d30-4ccf-4562-9edc-4c9d9014d803-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:35:59.989265 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.989191    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerID="d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8" exitCode=0
Jul 15 05:35:59.989414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.989284    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"
Jul 15 05:35:59.989414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.989284    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" event={"ID":"5bf86d30-4ccf-4562-9edc-4c9d9014d803","Type":"ContainerDied","Data":"d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8"}
Jul 15 05:35:59.989414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.989373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr" event={"ID":"5bf86d30-4ccf-4562-9edc-4c9d9014d803","Type":"ContainerDied","Data":"dd2b83b5169da1bb2aefa75df465519275f1b66e347a70f6aac7f863fe283239"}
Jul 15 05:35:59.989414 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.989398    2581 scope.go:117] "RemoveContainer" containerID="d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8"
Jul 15 05:35:59.998198 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:35:59.998183    2581 scope.go:117] "RemoveContainer" containerID="515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511"
Jul 15 05:36:00.006617 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:00.006602    2581 scope.go:117] "RemoveContainer" containerID="d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8"
Jul 15 05:36:00.006834 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:36:00.006817    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8\": container with ID starting with d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8 not found: ID does not exist" containerID="d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8"
Jul 15 05:36:00.006880 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:00.006842    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8"} err="failed to get container status \"d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8\": rpc error: code = NotFound desc = could not find container \"d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8\": container with ID starting with d4c0adaa43fca908042fab23c0ffbd745e1efdadb0d043e5953986c398a0c6a8 not found: ID does not exist"
Jul 15 05:36:00.006880 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:00.006858    2581 scope.go:117] "RemoveContainer" containerID="515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511"
Jul 15 05:36:00.007070 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:36:00.007056    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511\": container with ID starting with 515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511 not found: ID does not exist" containerID="515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511"
Jul 15 05:36:00.007108 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:00.007074    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511"} err="failed to get container status \"515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511\": rpc error: code = NotFound desc = could not find container \"515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511\": container with ID starting with 515d9e30b850351413cb423321978b1f1cc8cdf023fa644d5ff616534ff31511 not found: ID does not exist"
Jul 15 05:36:00.010779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:00.010757    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"]
Jul 15 05:36:00.014048 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:00.014029    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1fa2b35b-openshift-default-5f5565bb55-zxrpr"]
Jul 15 05:36:00.388180 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:00.388139    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" path="/var/lib/kubelet/pods/5bf86d30-4ccf-4562-9edc-4c9d9014d803/volumes"
Jul 15 05:36:15.759077 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.758991    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"]
Jul 15 05:36:15.759516 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759488    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy"
Jul 15 05:36:15.759516 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759510    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy"
Jul 15 05:36:15.759607 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759521    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b78879-4f4a-460a-a70f-bdb9c052fe2e" containerName="maas-api"
Jul 15 05:36:15.759607 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759530    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b78879-4f4a-460a-a70f-bdb9c052fe2e" containerName="maas-api"
Jul 15 05:36:15.759607 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759552    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy"
Jul 15 05:36:15.759607 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759580    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy"
Jul 15 05:36:15.759877 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759841    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy"
Jul 15 05:36:15.759877 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759865    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b78879-4f4a-460a-a70f-bdb9c052fe2e" containerName="maas-api"
Jul 15 05:36:15.760011 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.759881    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf86d30-4ccf-4562-9edc-4c9d9014d803" containerName="istio-proxy"
Jul 15 05:36:15.765115 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.765090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.767665 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.767613    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f84d43c4-openshift-default-dockercfg-h75s6\""
Jul 15 05:36:15.774767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.774744    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"]
Jul 15 05:36:15.859665 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.859627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-envoy\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.859839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.859701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47rfl\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-kube-api-access-47rfl\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.859839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.859745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b24a195e-fb43-4221-85ee-6a604d9ccca0-istiod-ca-cert\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.859839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.859819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-credential-socket\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.860012 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.859909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-certs\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.860012 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.859936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-data\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.860012 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.859980    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-token\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.860141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.860032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-socket\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.860141 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.860100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-podinfo\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.960834 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.960799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-credential-socket\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961002 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.960854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-certs\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961002 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.960957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-data\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961002 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.960992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-token\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-socket\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-podinfo\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961086    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-envoy\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47rfl\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-kube-api-access-47rfl\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b24a195e-fb43-4221-85ee-6a604d9ccca0-istiod-ca-cert\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-credential-socket\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-certs\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961535 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-data\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961611 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961591    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-socket\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.961851 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.961834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b24a195e-fb43-4221-85ee-6a604d9ccca0-istiod-ca-cert\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.963292 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.963263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-envoy\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.963550 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.963533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-podinfo\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.969157 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.969134    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-token\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:15.969258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:15.969167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47rfl\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-kube-api-access-47rfl\") pod \"e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") " pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:16.078953 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:16.078928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:16.216628 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:16.216608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"]
Jul 15 05:36:16.219111 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:36:16.219080    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24a195e_fb43_4221_85ee_6a604d9ccca0.slice/crio-50d8d488f5f3f455dea9d1b9917c316aba4e0909b481c7b8c19dced20387375d WatchSource:0}: Error finding container 50d8d488f5f3f455dea9d1b9917c316aba4e0909b481c7b8c19dced20387375d: Status 404 returned error can't find the container with id 50d8d488f5f3f455dea9d1b9917c316aba4e0909b481c7b8c19dced20387375d
Jul 15 05:36:16.221244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:16.221198    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:36:16.221324 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:16.221277    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:36:16.221324 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:16.221305    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:36:17.082543 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:17.082505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" event={"ID":"b24a195e-fb43-4221-85ee-6a604d9ccca0","Type":"ContainerStarted","Data":"ca29a5954a2735e685122dbd5b162264b18bbbaae94b066f0bd427979987cd20"}
Jul 15 05:36:17.082908 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:17.082560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" event={"ID":"b24a195e-fb43-4221-85ee-6a604d9ccca0","Type":"ContainerStarted","Data":"50d8d488f5f3f455dea9d1b9917c316aba4e0909b481c7b8c19dced20387375d"}
Jul 15 05:36:17.102727 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:17.102673    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" podStartSLOduration=2.102659029 podStartE2EDuration="2.102659029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:36:17.100660723 +0000 UTC m=+2255.268257395" watchObservedRunningTime="2026-07-15 05:36:17.102659029 +0000 UTC m=+2255.270255700"
Jul 15 05:36:18.080053 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:18.080013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:18.085419 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:18.085396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:18.087772 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:18.087752    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:18.088520 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:18.088500    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:19.185615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.185561    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"]
Jul 15 05:36:19.188798 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.188776    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.191621 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.191598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f84d43c4-serving-cert\""
Jul 15 05:36:19.201105 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.201077    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"]
Jul 15 05:36:19.292773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.292736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-openshift-service-ca\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.292977 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.292901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-maas-api-tls\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.292977 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.292937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gccld\" (UniqueName: \"kubernetes.io/projected/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-kube-api-access-gccld\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.394127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.394094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-openshift-service-ca\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.394331 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.394252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-maas-api-tls\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.394331 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.394283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gccld\" (UniqueName: \"kubernetes.io/projected/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-kube-api-access-gccld\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.394886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.394860    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-openshift-service-ca\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.396640 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.396614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-maas-api-tls\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.401622 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.401601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gccld\" (UniqueName: \"kubernetes.io/projected/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-kube-api-access-gccld\") pod \"maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.502699 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.502626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:19.643150 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:19.643122    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"]
Jul 15 05:36:19.644836 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:36:19.644811    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61189b51_8a8e_4f94_bbe7_f4c44f7f2093.slice/crio-5ad9a07c83ac1177f884cf4aeea37844a0f9887de24726c45a426787fce02b7a WatchSource:0}: Error finding container 5ad9a07c83ac1177f884cf4aeea37844a0f9887de24726c45a426787fce02b7a: Status 404 returned error can't find the container with id 5ad9a07c83ac1177f884cf4aeea37844a0f9887de24726c45a426787fce02b7a
Jul 15 05:36:20.100648 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:20.100621    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz" event={"ID":"61189b51-8a8e-4f94-bbe7-f4c44f7f2093","Type":"ContainerStarted","Data":"5ad9a07c83ac1177f884cf4aeea37844a0f9887de24726c45a426787fce02b7a"}
Jul 15 05:36:21.106997 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:21.106960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz" event={"ID":"61189b51-8a8e-4f94-bbe7-f4c44f7f2093","Type":"ContainerStarted","Data":"b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f"}
Jul 15 05:36:21.107442 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:21.107023    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:21.124616 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:21.124568    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz" podStartSLOduration=1.707580803 podStartE2EDuration="2.124552012s" totalImagesPullingTime="416.971209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:36:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:36:21.12248257 +0000 UTC m=+2259.290079242" watchObservedRunningTime="2026-07-15 05:36:21.124552012 +0000 UTC m=+2259.292148684"
Jul 15 05:36:27.117975 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:27.117930    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:36:32.167750 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:32.167713    2581 generic.go:356] "Generic (PLEG): container finished" podID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerID="ca29a5954a2735e685122dbd5b162264b18bbbaae94b066f0bd427979987cd20" exitCode=0
Jul 15 05:36:32.168166 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:32.167788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" event={"ID":"b24a195e-fb43-4221-85ee-6a604d9ccca0","Type":"ContainerDied","Data":"ca29a5954a2735e685122dbd5b162264b18bbbaae94b066f0bd427979987cd20"}
Jul 15 05:36:32.168166 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:32.168164    2581 scope.go:117] "RemoveContainer" containerID="ca29a5954a2735e685122dbd5b162264b18bbbaae94b066f0bd427979987cd20"
Jul 15 05:36:32.169173 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:32.169141    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:36:32.169321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:32.169236    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:36:32.169321 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:32.169271    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:36:33.088299 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:33.088256    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:33.175564 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:33.175519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" event={"ID":"b24a195e-fb43-4221-85ee-6a604d9ccca0","Type":"ContainerStarted","Data":"ae21caa28d5bec00a4a18698a999b25943b14351e7bc2f68f1344d05a0a5d4cd"}
Jul 15 05:36:34.081076 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:34.079741    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:34.081760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:34.081429    2581 patch_prober.go:28] interesting pod/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 15 05:36:34.081760 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:34.081498    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 15 05:36:35.079503 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:35.079463    2581 patch_prober.go:28] interesting pod/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 15 05:36:35.080528 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:35.080063    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 15 05:36:36.080395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:36.080360    2581 patch_prober.go:28] interesting pod/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 15 05:36:36.080973 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:36.080941    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 15 05:36:37.084029 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:37.083985    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:37.084517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:37.084394    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:36:37.085112 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:36:37.085090    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:37:12.585112 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.585076    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"]
Jul 15 05:37:12.587097 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.587044    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz" podUID="61189b51-8a8e-4f94-bbe7-f4c44f7f2093" containerName="maas-api" containerID="cri-o://b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f" gracePeriod=30
Jul 15 05:37:12.843471 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.843417    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:37:12.989379 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.989350    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gccld\" (UniqueName: \"kubernetes.io/projected/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-kube-api-access-gccld\") pod \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") "
Jul 15 05:37:12.989532 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.989468    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-openshift-service-ca\") pod \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") "
Jul 15 05:37:12.989532 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.989496    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-maas-api-tls\") pod \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\" (UID: \"61189b51-8a8e-4f94-bbe7-f4c44f7f2093\") "
Jul 15 05:37:12.989864 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.989836    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "61189b51-8a8e-4f94-bbe7-f4c44f7f2093" (UID: "61189b51-8a8e-4f94-bbe7-f4c44f7f2093"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:37:12.991495 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.991468    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-kube-api-access-gccld" (OuterVolumeSpecName: "kube-api-access-gccld") pod "61189b51-8a8e-4f94-bbe7-f4c44f7f2093" (UID: "61189b51-8a8e-4f94-bbe7-f4c44f7f2093"). InnerVolumeSpecName "kube-api-access-gccld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:37:12.991495 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:12.991482    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "61189b51-8a8e-4f94-bbe7-f4c44f7f2093" (UID: "61189b51-8a8e-4f94-bbe7-f4c44f7f2093"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:37:13.090718 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.090690    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gccld\" (UniqueName: \"kubernetes.io/projected/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-kube-api-access-gccld\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:37:13.090718 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.090716    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:37:13.090863 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.090728    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61189b51-8a8e-4f94-bbe7-f4c44f7f2093-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:37:13.395534 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.395497    2581 generic.go:356] "Generic (PLEG): container finished" podID="61189b51-8a8e-4f94-bbe7-f4c44f7f2093" containerID="b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f" exitCode=0
Jul 15 05:37:13.395696 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.395535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz" event={"ID":"61189b51-8a8e-4f94-bbe7-f4c44f7f2093","Type":"ContainerDied","Data":"b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f"}
Jul 15 05:37:13.395696 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.395553    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"
Jul 15 05:37:13.395696 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.395601    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz" event={"ID":"61189b51-8a8e-4f94-bbe7-f4c44f7f2093","Type":"ContainerDied","Data":"5ad9a07c83ac1177f884cf4aeea37844a0f9887de24726c45a426787fce02b7a"}
Jul 15 05:37:13.395696 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.395625    2581 scope.go:117] "RemoveContainer" containerID="b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f"
Jul 15 05:37:13.404948 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.404933    2581 scope.go:117] "RemoveContainer" containerID="b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f"
Jul 15 05:37:13.405238 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:37:13.405215    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f\": container with ID starting with b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f not found: ID does not exist" containerID="b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f"
Jul 15 05:37:13.405329 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.405247    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f"} err="failed to get container status \"b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f\": rpc error: code = NotFound desc = could not find container \"b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f\": container with ID starting with b85a791216d0018571d89d60cd0f1f8e0f05f563df53917f0c740592fb71cf3f not found: ID does not exist"
Jul 15 05:37:13.415874 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.415850    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"]
Jul 15 05:37:13.418068 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:13.418049    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f84d43c4-56d6978749-wwvxz"]
Jul 15 05:37:14.388886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:37:14.388852    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61189b51-8a8e-4f94-bbe7-f4c44f7f2093" path="/var/lib/kubelet/pods/61189b51-8a8e-4f94-bbe7-f4c44f7f2093/volumes"
Jul 15 05:38:19.604445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:19.604397    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"]
Jul 15 05:38:19.605214 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:19.604660    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy" containerID="cri-o://ae21caa28d5bec00a4a18698a999b25943b14351e7bc2f68f1344d05a0a5d4cd" gracePeriod=30
Jul 15 05:38:22.086894 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:22.086854    2581 patch_prober.go:28] interesting pod/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 05:38:22.087288 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:22.086923    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 05:38:24.763125 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:24.763087    2581 generic.go:356] "Generic (PLEG): container finished" podID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerID="ae21caa28d5bec00a4a18698a999b25943b14351e7bc2f68f1344d05a0a5d4cd" exitCode=0
Jul 15 05:38:24.763647 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:24.763181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" event={"ID":"b24a195e-fb43-4221-85ee-6a604d9ccca0","Type":"ContainerDied","Data":"ae21caa28d5bec00a4a18698a999b25943b14351e7bc2f68f1344d05a0a5d4cd"}
Jul 15 05:38:24.763647 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:24.763264    2581 scope.go:117] "RemoveContainer" containerID="ca29a5954a2735e685122dbd5b162264b18bbbaae94b066f0bd427979987cd20"
Jul 15 05:38:24.869552 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:24.869490    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:38:25.018057 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018029    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-podinfo\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018255 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-credential-socket\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018255 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018136    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47rfl\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-kube-api-access-47rfl\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018255 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-token\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018255 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018231    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-data\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018283    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-socket\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018337    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-envoy\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018397    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:38:25.018572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018424    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b24a195e-fb43-4221-85ee-6a604d9ccca0-istiod-ca-cert\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018498    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-certs\") pod \"b24a195e-fb43-4221-85ee-6a604d9ccca0\" (UID: \"b24a195e-fb43-4221-85ee-6a604d9ccca0\") "
Jul 15 05:38:25.018572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:38:25.018873 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018774    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b24a195e-fb43-4221-85ee-6a604d9ccca0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:38:25.018950 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.018920    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:38:25.019042 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.019015    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.019042 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.019044    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b24a195e-fb43-4221-85ee-6a604d9ccca0-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.019272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.019058    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.019272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.019072    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.019272 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.019149    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-data" (OuterVolumeSpecName: "istio-data") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:38:25.020593 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.020569    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:38:25.021476 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.021444    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-kube-api-access-47rfl" (OuterVolumeSpecName: "kube-api-access-47rfl") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "kube-api-access-47rfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:38:25.021476 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.021444    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:38:25.022095 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.022068    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-token" (OuterVolumeSpecName: "istio-token") pod "b24a195e-fb43-4221-85ee-6a604d9ccca0" (UID: "b24a195e-fb43-4221-85ee-6a604d9ccca0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:38:25.120172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.120093    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.120172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.120121    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47rfl\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-kube-api-access-47rfl\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.120172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.120133    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.120172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.120141    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.120172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.120149    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b24a195e-fb43-4221-85ee-6a604d9ccca0-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:38:25.770329 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.770300    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"
Jul 15 05:38:25.770726 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.770300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf" event={"ID":"b24a195e-fb43-4221-85ee-6a604d9ccca0","Type":"ContainerDied","Data":"50d8d488f5f3f455dea9d1b9917c316aba4e0909b481c7b8c19dced20387375d"}
Jul 15 05:38:25.770726 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.770485    2581 scope.go:117] "RemoveContainer" containerID="ae21caa28d5bec00a4a18698a999b25943b14351e7bc2f68f1344d05a0a5d4cd"
Jul 15 05:38:25.800301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.800266    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"]
Jul 15 05:38:25.803933 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:25.803907    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f84d43c4-openshift-default-695885fddc-84wzf"]
Jul 15 05:38:26.388677 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:38:26.388644    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" path="/var/lib/kubelet/pods/b24a195e-fb43-4221-85ee-6a604d9ccca0/volumes"
Jul 15 05:39:14.037453 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.037377    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"]
Jul 15 05:39:14.037977 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.037961    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy"
Jul 15 05:39:14.038021 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.037981    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy"
Jul 15 05:39:14.038056 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.038048    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61189b51-8a8e-4f94-bbe7-f4c44f7f2093" containerName="maas-api"
Jul 15 05:39:14.038089 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.038058    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="61189b51-8a8e-4f94-bbe7-f4c44f7f2093" containerName="maas-api"
Jul 15 05:39:14.038089 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.038070    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy"
Jul 15 05:39:14.038089 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.038079    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy"
Jul 15 05:39:14.038182 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.038171    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy"
Jul 15 05:39:14.038245 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.038182    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="61189b51-8a8e-4f94-bbe7-f4c44f7f2093" containerName="maas-api"
Jul 15 05:39:14.038245 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.038194    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b24a195e-fb43-4221-85ee-6a604d9ccca0" containerName="istio-proxy"
Jul 15 05:39:14.040916 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.040893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.043522 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.043501    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-891b7e-openshift-default-dockercfg-6pwqc\""
Jul 15 05:39:14.051460 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.051439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"]
Jul 15 05:39:14.128667 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-certs\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.128834 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-data\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.128834 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn4sh\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-kube-api-access-cn4sh\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.128834 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-socket\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.128958 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-credential-socket\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.128958 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-podinfo\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.128958 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-envoy\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.129068 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.128991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istiod-ca-cert\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.129068 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.129032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-token\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.229643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.229609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-token\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.229804 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.229656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-certs\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.229804 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.229768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-data\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.229929 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.229814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn4sh\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-kube-api-access-cn4sh\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.229929 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.229875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-socket\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.229929 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.229909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-credential-socket\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.229932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-podinfo\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.230006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-envoy\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.230014    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-certs\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230099 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.230049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istiod-ca-cert\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230293 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.230128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-data\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230293 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.230198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-socket\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230293 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.230285    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-credential-socket\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.230785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.230763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istiod-ca-cert\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.232294 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.232274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-envoy\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.232536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.232516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-podinfo\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.237555 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.237534    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-token\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.238036 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.238009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn4sh\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-kube-api-access-cn4sh\") pod \"e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") " pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.351815 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.351791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:14.490866 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.490836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"]
Jul 15 05:39:14.492340 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:39:14.492313    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e509e3_10b4_4cea_904d_fe0dbaf52e03.slice/crio-244977c3a3ce5c7f4e0d4c737d67859d5949acb226fbf6ec7b9240919fcf33cb WatchSource:0}: Error finding container 244977c3a3ce5c7f4e0d4c737d67859d5949acb226fbf6ec7b9240919fcf33cb: Status 404 returned error can't find the container with id 244977c3a3ce5c7f4e0d4c737d67859d5949acb226fbf6ec7b9240919fcf33cb
Jul 15 05:39:14.494628 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.494590    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:14.494712 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.494659    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:14.494712 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:14.494685    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:15.030430 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:15.030396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" event={"ID":"71e509e3-10b4-4cea-904d-fe0dbaf52e03","Type":"ContainerStarted","Data":"841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f"}
Jul 15 05:39:15.030608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:15.030453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" event={"ID":"71e509e3-10b4-4cea-904d-fe0dbaf52e03","Type":"ContainerStarted","Data":"244977c3a3ce5c7f4e0d4c737d67859d5949acb226fbf6ec7b9240919fcf33cb"}
Jul 15 05:39:15.050504 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:15.050456    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podStartSLOduration=1.050441663 podStartE2EDuration="1.050441663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:39:15.049494497 +0000 UTC m=+2433.217091168" watchObservedRunningTime="2026-07-15 05:39:15.050441663 +0000 UTC m=+2433.218038335"
Jul 15 05:39:15.352835 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:15.352808    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:15.357548 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:15.357527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:16.035640 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:16.035607    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:16.036535 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:16.036509    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:17.120395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.120352    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"]
Jul 15 05:39:17.123519 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.123500    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.125962 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.125942    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-891b7e-serving-cert\""
Jul 15 05:39:17.134640 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.134614    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"]
Jul 15 05:39:17.257412 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.257382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vllfd\" (UniqueName: \"kubernetes.io/projected/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-kube-api-access-vllfd\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.257604 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.257434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.257604 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.257545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.358761 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.358729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.358922 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.358789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vllfd\" (UniqueName: \"kubernetes.io/projected/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-kube-api-access-vllfd\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.358922 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.358829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.358922 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:39:17.358895    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-serving-cert: secret "maas-api-e2e-inf-a-891b7e-serving-cert" not found
Jul 15 05:39:17.359087 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:39:17.358969    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls podName:36d3a12f-0ad2-4228-b5e4-c9642e0431fb nodeName:}" failed. No retries permitted until 2026-07-15 05:39:17.858947431 +0000 UTC m=+2436.026544096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls") pod "maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh" (UID: "36d3a12f-0ad2-4228-b5e4-c9642e0431fb") : secret "maas-api-e2e-inf-a-891b7e-serving-cert" not found
Jul 15 05:39:17.359495 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.359477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.369896 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.369877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vllfd\" (UniqueName: \"kubernetes.io/projected/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-kube-api-access-vllfd\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.863851 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.863814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:17.866304 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:17.866280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls\") pod \"maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:18.035126 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:18.035090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:18.182406 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:18.182384    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"]
Jul 15 05:39:18.184641 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:39:18.184613    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d3a12f_0ad2_4228_b5e4_c9642e0431fb.slice/crio-edc08eb2ed563548c8ce2b336c50ddc52e80b4fae1a8b1ab5d0f93395b3e345d WatchSource:0}: Error finding container edc08eb2ed563548c8ce2b336c50ddc52e80b4fae1a8b1ab5d0f93395b3e345d: Status 404 returned error can't find the container with id edc08eb2ed563548c8ce2b336c50ddc52e80b4fae1a8b1ab5d0f93395b3e345d
Jul 15 05:39:19.055632 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:19.055550    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh" event={"ID":"36d3a12f-0ad2-4228-b5e4-c9642e0431fb","Type":"ContainerStarted","Data":"1049044714ed5857d18ffd98324bf67d02499ccdd57a344591c128179bb64a09"}
Jul 15 05:39:19.055782 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:19.055641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh" event={"ID":"36d3a12f-0ad2-4228-b5e4-c9642e0431fb","Type":"ContainerStarted","Data":"edc08eb2ed563548c8ce2b336c50ddc52e80b4fae1a8b1ab5d0f93395b3e345d"}
Jul 15 05:39:19.055782 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:19.055679    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:19.071331 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:19.071281    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh" podStartSLOduration=1.4710196660000001 podStartE2EDuration="2.07126376s" totalImagesPullingTime="600.244094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:39:19.070568167 +0000 UTC m=+2437.238164841" watchObservedRunningTime="2026-07-15 05:39:19.07126376 +0000 UTC m=+2437.238860436"
Jul 15 05:39:22.157008 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.156975    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"]
Jul 15 05:39:22.160567 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.160525    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.163220 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.163175    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-de675f-openshift-default-dockercfg-vfx48\""
Jul 15 05:39:22.173764 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.173743    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"]
Jul 15 05:39:22.308989 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.308935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-socket\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309185 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4pg8\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-kube-api-access-w4pg8\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309185 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-certs\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309346 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835232b-78e6-43a9-9d88-155cd947012a-istio-podinfo\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309346 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-envoy\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309346 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-istio-token\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309497 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-data\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309497 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-credential-socket\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.309497 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.309435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835232b-78e6-43a9-9d88-155cd947012a-istiod-ca-cert\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835232b-78e6-43a9-9d88-155cd947012a-istiod-ca-cert\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-socket\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4pg8\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-kube-api-access-w4pg8\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410403 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-certs\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410403 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835232b-78e6-43a9-9d88-155cd947012a-istio-podinfo\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410403 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-envoy\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410403 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-istio-token\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-data\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-credential-socket\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-socket\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.410605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.410593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-certs\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.413443 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.411119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835232b-78e6-43a9-9d88-155cd947012a-istiod-ca-cert\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.413443 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.411322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-data\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.413443 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.411474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-credential-socket\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.415258 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.414577    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835232b-78e6-43a9-9d88-155cd947012a-istio-podinfo\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.420748 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.420657    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-envoy\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.422241 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.422194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-istio-token\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.423524 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.423499    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4pg8\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-kube-api-access-w4pg8\") pod \"e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") " pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.472182 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.472159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:22.613490 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.612789    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"]
Jul 15 05:39:22.617841 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:39:22.617813    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode835232b_78e6_43a9_9d88_155cd947012a.slice/crio-cd8ce60a2bf4a03a16f8a4466ceaabb71ee5fc5eef5e3cc5d3fd427d9d3ab3a5 WatchSource:0}: Error finding container cd8ce60a2bf4a03a16f8a4466ceaabb71ee5fc5eef5e3cc5d3fd427d9d3ab3a5: Status 404 returned error can't find the container with id cd8ce60a2bf4a03a16f8a4466ceaabb71ee5fc5eef5e3cc5d3fd427d9d3ab3a5
Jul 15 05:39:22.620292 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.620249    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:22.620375 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.620344    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:22.620432 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:22.620400    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:23.080638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:23.080598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" event={"ID":"e835232b-78e6-43a9-9d88-155cd947012a","Type":"ContainerStarted","Data":"041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e"}
Jul 15 05:39:23.080793 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:23.080669    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" event={"ID":"e835232b-78e6-43a9-9d88-155cd947012a","Type":"ContainerStarted","Data":"cd8ce60a2bf4a03a16f8a4466ceaabb71ee5fc5eef5e3cc5d3fd427d9d3ab3a5"}
Jul 15 05:39:23.099830 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:23.099786    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podStartSLOduration=1.099772774 podStartE2EDuration="1.099772774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:39:23.09797478 +0000 UTC m=+2441.265571465" watchObservedRunningTime="2026-07-15 05:39:23.099772774 +0000 UTC m=+2441.267369443"
Jul 15 05:39:23.473380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:23.473305    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:23.474655 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:23.474630    2581 patch_prober.go:28] interesting pod/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 15 05:39:23.474765 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:23.474684    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 15 05:39:24.478096 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:24.478064    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:25.066542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.066516    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:39:25.090831 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.090797    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:25.092096 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.092073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:25.110154 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.110129    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"]
Jul 15 05:39:25.113134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.113113    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.115521 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.115496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-de675f-serving-cert\""
Jul 15 05:39:25.129021 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.128996    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"]
Jul 15 05:39:25.238348 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.238316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3376b29f-aee7-4e7f-a86e-ef9fb338e240-openshift-service-ca\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.238784 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.238759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.238908 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.238870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5z2\" (UniqueName: \"kubernetes.io/projected/3376b29f-aee7-4e7f-a86e-ef9fb338e240-kube-api-access-ns5z2\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.340456 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.340366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.340456 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.340445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5z2\" (UniqueName: \"kubernetes.io/projected/3376b29f-aee7-4e7f-a86e-ef9fb338e240-kube-api-access-ns5z2\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.340650 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:39:25.340514    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-serving-cert: secret "maas-api-e2e-inf-b-de675f-serving-cert" not found
Jul 15 05:39:25.340650 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.340535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3376b29f-aee7-4e7f-a86e-ef9fb338e240-openshift-service-ca\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.340650 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:39:25.340575    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls podName:3376b29f-aee7-4e7f-a86e-ef9fb338e240 nodeName:}" failed. No retries permitted until 2026-07-15 05:39:25.840557603 +0000 UTC m=+2444.008154256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls") pod "maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" (UID: "3376b29f-aee7-4e7f-a86e-ef9fb338e240") : secret "maas-api-e2e-inf-b-de675f-serving-cert" not found
Jul 15 05:39:25.341106 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.341089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3376b29f-aee7-4e7f-a86e-ef9fb338e240-openshift-service-ca\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.349857 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.349826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5z2\" (UniqueName: \"kubernetes.io/projected/3376b29f-aee7-4e7f-a86e-ef9fb338e240-kube-api-access-ns5z2\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.847022 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.846987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:25.850312 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:25.850284    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls\") pod \"maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:26.027671 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:26.027641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:26.164943 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:26.164910    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"]
Jul 15 05:39:26.166315 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:39:26.166286    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3376b29f_aee7_4e7f_a86e_ef9fb338e240.slice/crio-188096d774dcabe70708f9e5d55598047b01853fc4de6f7e3b33e1644ee62e2a WatchSource:0}: Error finding container 188096d774dcabe70708f9e5d55598047b01853fc4de6f7e3b33e1644ee62e2a: Status 404 returned error can't find the container with id 188096d774dcabe70708f9e5d55598047b01853fc4de6f7e3b33e1644ee62e2a
Jul 15 05:39:27.105471 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:27.105431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" event={"ID":"3376b29f-aee7-4e7f-a86e-ef9fb338e240","Type":"ContainerStarted","Data":"1f2d2f09c8d8b9cc48cf78707b2b959b31cb42eeaee094956d96f43364802788"}
Jul 15 05:39:27.105924 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:27.105493    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" event={"ID":"3376b29f-aee7-4e7f-a86e-ef9fb338e240","Type":"ContainerStarted","Data":"188096d774dcabe70708f9e5d55598047b01853fc4de6f7e3b33e1644ee62e2a"}
Jul 15 05:39:27.105924 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:27.105745    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:27.138188 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:27.138134    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" podStartSLOduration=1.797207778 podStartE2EDuration="2.138117395s" totalImagesPullingTime="340.909617ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:39:27.134939508 +0000 UTC m=+2445.302536191" watchObservedRunningTime="2026-07-15 05:39:27.138117395 +0000 UTC m=+2445.305714094"
Jul 15 05:39:32.077819 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:32.077400    2581 patch_prober.go:28] interesting pod/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 15 05:39:32.077819 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:32.077489    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 15 05:39:33.116683 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:33.116651    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:39:34.143925 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:34.143847    2581 generic.go:356] "Generic (PLEG): container finished" podID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerID="841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f" exitCode=0
Jul 15 05:39:34.144365 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:34.143936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" event={"ID":"71e509e3-10b4-4cea-904d-fe0dbaf52e03","Type":"ContainerDied","Data":"841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f"}
Jul 15 05:39:34.144447 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:34.144366    2581 scope.go:117] "RemoveContainer" containerID="841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f"
Jul 15 05:39:34.145541 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:34.145512    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:34.145620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:34.145580    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:34.145620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:34.145613    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:35.151130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:35.151095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" event={"ID":"71e509e3-10b4-4cea-904d-fe0dbaf52e03","Type":"ContainerStarted","Data":"8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90"}
Jul 15 05:39:35.352306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:35.352269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:35.352478 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:35.352409    2581 patch_prober.go:28] interesting pod/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 15 05:39:35.352478 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:35.352460    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 15 05:39:36.352581 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:36.352528    2581 patch_prober.go:28] interesting pod/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 15 05:39:36.353033 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:36.352601    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 15 05:39:37.352565 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:37.352519    2581 patch_prober.go:28] interesting pod/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 15 05:39:37.352770 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:37.352591    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 15 05:39:38.352732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:38.352694    2581 patch_prober.go:28] interesting pod/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 15 05:39:38.352903 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:38.352747    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 15 05:39:39.352682 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:39.352615    2581 patch_prober.go:28] interesting pod/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 15 05:39:39.352886 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:39.352696    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 15 05:39:40.092916 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.092874    2581 patch_prober.go:28] interesting pod/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 15 05:39:40.093359 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.092932    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 15 05:39:40.182320 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.182286    2581 generic.go:356] "Generic (PLEG): container finished" podID="e835232b-78e6-43a9-9d88-155cd947012a" containerID="041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e" exitCode=0
Jul 15 05:39:40.182483 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.182364    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" event={"ID":"e835232b-78e6-43a9-9d88-155cd947012a","Type":"ContainerDied","Data":"041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e"}
Jul 15 05:39:40.182789 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.182770    2581 scope.go:117] "RemoveContainer" containerID="041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e"
Jul 15 05:39:40.183971 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.183938    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:40.184093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.184011    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:40.184093 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.184044    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:39:40.356708 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.356626    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:40.356910 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.356893    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:40.357967 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:40.357945    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:39:41.193897 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:41.193852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" event={"ID":"e835232b-78e6-43a9-9d88-155cd947012a","Type":"ContainerStarted","Data":"a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23"}
Jul 15 05:39:41.473333 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:41.473250    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:41.473578 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:41.473463    2581 patch_prober.go:28] interesting pod/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 15 05:39:41.473737 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:41.473713    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 15 05:39:42.472781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:42.472743    2581 patch_prober.go:28] interesting pod/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 15 05:39:42.473154 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:42.472808    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 15 05:39:43.473656 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:43.473615    2581 patch_prober.go:28] interesting pod/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 15 05:39:43.474132 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:43.473687    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 15 05:39:44.473611 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:44.473545    2581 patch_prober.go:28] interesting pod/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 15 05:39:44.474011 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:44.473642    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 15 05:39:45.476537 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:45.476510    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:45.477019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:45.476762    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:39:45.477637 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:39:45.477613    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:40:51.308007 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.307974    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"]
Jul 15 05:40:51.308638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.308356    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh" podUID="36d3a12f-0ad2-4228-b5e4-c9642e0431fb" containerName="maas-api" containerID="cri-o://1049044714ed5857d18ffd98324bf67d02499ccdd57a344591c128179bb64a09" gracePeriod=30
Jul 15 05:40:51.565072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.565007    2581 generic.go:356] "Generic (PLEG): container finished" podID="36d3a12f-0ad2-4228-b5e4-c9642e0431fb" containerID="1049044714ed5857d18ffd98324bf67d02499ccdd57a344591c128179bb64a09" exitCode=0
Jul 15 05:40:51.565189 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.565081    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh" event={"ID":"36d3a12f-0ad2-4228-b5e4-c9642e0431fb","Type":"ContainerDied","Data":"1049044714ed5857d18ffd98324bf67d02499ccdd57a344591c128179bb64a09"}
Jul 15 05:40:51.565189 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.565128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh" event={"ID":"36d3a12f-0ad2-4228-b5e4-c9642e0431fb","Type":"ContainerDied","Data":"edc08eb2ed563548c8ce2b336c50ddc52e80b4fae1a8b1ab5d0f93395b3e345d"}
Jul 15 05:40:51.565189 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.565145    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edc08eb2ed563548c8ce2b336c50ddc52e80b4fae1a8b1ab5d0f93395b3e345d"
Jul 15 05:40:51.565503 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.565488    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:40:51.623388 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.623368    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vllfd\" (UniqueName: \"kubernetes.io/projected/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-kube-api-access-vllfd\") pod \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") "
Jul 15 05:40:51.623525 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.623416    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls\") pod \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") "
Jul 15 05:40:51.623525 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.623447    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-openshift-service-ca\") pod \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\" (UID: \"36d3a12f-0ad2-4228-b5e4-c9642e0431fb\") "
Jul 15 05:40:51.623863 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.623841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36d3a12f-0ad2-4228-b5e4-c9642e0431fb" (UID: "36d3a12f-0ad2-4228-b5e4-c9642e0431fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:40:51.625498 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.625478    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36d3a12f-0ad2-4228-b5e4-c9642e0431fb" (UID: "36d3a12f-0ad2-4228-b5e4-c9642e0431fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:40:51.625569 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.625531    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-kube-api-access-vllfd" (OuterVolumeSpecName: "kube-api-access-vllfd") pod "36d3a12f-0ad2-4228-b5e4-c9642e0431fb" (UID: "36d3a12f-0ad2-4228-b5e4-c9642e0431fb"). InnerVolumeSpecName "kube-api-access-vllfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:40:51.724504 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.724479    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vllfd\" (UniqueName: \"kubernetes.io/projected/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-kube-api-access-vllfd\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:40:51.724504 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.724503    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:40:51.724661 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:51.724513    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d3a12f-0ad2-4228-b5e4-c9642e0431fb-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:40:52.569660 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:52.569630    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"
Jul 15 05:40:52.586067 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:52.586038    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"]
Jul 15 05:40:52.588072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:52.588049    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-891b7e-865485bff9-l6dvh"]
Jul 15 05:40:54.388271 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:40:54.388235    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36d3a12f-0ad2-4228-b5e4-c9642e0431fb" path="/var/lib/kubelet/pods/36d3a12f-0ad2-4228-b5e4-c9642e0431fb/volumes"
Jul 15 05:41:06.678040 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:06.677999    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"]
Jul 15 05:41:06.678536 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:06.678262    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" containerID="cri-o://8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90" gracePeriod=30
Jul 15 05:41:10.358961 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:10.358925    2581 patch_prober.go:28] interesting pod/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 05:41:10.359351 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:10.358987    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 05:41:11.941572 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:11.941517    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:41:12.009711 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009689    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-socket\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.009839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-credential-socket\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.009839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009748    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-token\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.009839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009792    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-data\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.009839 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009818    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-envoy\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.010072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009865    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn4sh\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-kube-api-access-cn4sh\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.010072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009906    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-certs\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.010072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009940    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istiod-ca-cert\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.010072 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009977    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-podinfo\") pod \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\" (UID: \"71e509e3-10b4-4cea-904d-fe0dbaf52e03\") "
Jul 15 05:41:12.010380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.009972    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:12.010380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.010005    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:12.010380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.010163    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:12.010380 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.010346    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:41:12.010566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.010401    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.010566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.010417    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.010566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.010429    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.010798 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.010772    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-data" (OuterVolumeSpecName: "istio-data") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:12.012091 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.012057    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:41:12.012166 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.012144    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:12.012504 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.012479    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-kube-api-access-cn4sh" (OuterVolumeSpecName: "kube-api-access-cn4sh") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "kube-api-access-cn4sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:41:12.012620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.012600    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-token" (OuterVolumeSpecName: "istio-token") pod "71e509e3-10b4-4cea-904d-fe0dbaf52e03" (UID: "71e509e3-10b4-4cea-904d-fe0dbaf52e03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:41:12.110825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.110806    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.110825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.110826    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.110956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.110834    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.110956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.110843    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn4sh\" (UniqueName: \"kubernetes.io/projected/71e509e3-10b4-4cea-904d-fe0dbaf52e03-kube-api-access-cn4sh\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.110956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.110853    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.110956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.110864    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e509e3-10b4-4cea-904d-fe0dbaf52e03-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:12.676968 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.676890    2581 generic.go:356] "Generic (PLEG): container finished" podID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerID="8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90" exitCode=0
Jul 15 05:41:12.676968 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.676946    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"
Jul 15 05:41:12.677134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.676972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" event={"ID":"71e509e3-10b4-4cea-904d-fe0dbaf52e03","Type":"ContainerDied","Data":"8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90"}
Jul 15 05:41:12.677134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.677026    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm" event={"ID":"71e509e3-10b4-4cea-904d-fe0dbaf52e03","Type":"ContainerDied","Data":"244977c3a3ce5c7f4e0d4c737d67859d5949acb226fbf6ec7b9240919fcf33cb"}
Jul 15 05:41:12.677134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.677056    2581 scope.go:117] "RemoveContainer" containerID="8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90"
Jul 15 05:41:12.685870 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.685856    2581 scope.go:117] "RemoveContainer" containerID="841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f"
Jul 15 05:41:12.693570 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.693552    2581 scope.go:117] "RemoveContainer" containerID="8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90"
Jul 15 05:41:12.693834 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:41:12.693814    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90\": container with ID starting with 8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90 not found: ID does not exist" containerID="8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90"
Jul 15 05:41:12.693905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.693843    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90"} err="failed to get container status \"8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90\": rpc error: code = NotFound desc = could not find container \"8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90\": container with ID starting with 8a10d9ac04d5e3ed11e70781706e5cad34691892278c1509d5620e26b5b10a90 not found: ID does not exist"
Jul 15 05:41:12.693905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.693863    2581 scope.go:117] "RemoveContainer" containerID="841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f"
Jul 15 05:41:12.694214 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:41:12.694156    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f\": container with ID starting with 841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f not found: ID does not exist" containerID="841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f"
Jul 15 05:41:12.694214 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.694190    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f"} err="failed to get container status \"841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f\": rpc error: code = NotFound desc = could not find container \"841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f\": container with ID starting with 841541836eea74c6b0f74d334fa7fe244dfd3834ff92daa29e6a4bc520b1f85f not found: ID does not exist"
Jul 15 05:41:12.696866 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.696847    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"]
Jul 15 05:41:12.699658 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:12.699638    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-891b7e-openshift-default-775fd99ccc-8vlsm"]
Jul 15 05:41:14.388998 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:14.388963    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" path="/var/lib/kubelet/pods/71e509e3-10b4-4cea-904d-fe0dbaf52e03/volumes"
Jul 15 05:41:17.515470 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:17.515432    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"]
Jul 15 05:41:17.516298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:17.516196    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" podUID="3376b29f-aee7-4e7f-a86e-ef9fb338e240" containerName="maas-api" containerID="cri-o://1f2d2f09c8d8b9cc48cf78707b2b959b31cb42eeaee094956d96f43364802788" gracePeriod=30
Jul 15 05:41:18.111808 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.111768    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" podUID="3376b29f-aee7-4e7f-a86e-ef9fb338e240" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.70:8443/health\": dial tcp 10.134.0.70:8443: connect: connection refused"
Jul 15 05:41:18.712485 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.712454    2581 generic.go:356] "Generic (PLEG): container finished" podID="3376b29f-aee7-4e7f-a86e-ef9fb338e240" containerID="1f2d2f09c8d8b9cc48cf78707b2b959b31cb42eeaee094956d96f43364802788" exitCode=0
Jul 15 05:41:18.712919 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.712526    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" event={"ID":"3376b29f-aee7-4e7f-a86e-ef9fb338e240","Type":"ContainerDied","Data":"1f2d2f09c8d8b9cc48cf78707b2b959b31cb42eeaee094956d96f43364802788"}
Jul 15 05:41:18.763913 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.763891    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:41:18.867909 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.867883    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns5z2\" (UniqueName: \"kubernetes.io/projected/3376b29f-aee7-4e7f-a86e-ef9fb338e240-kube-api-access-ns5z2\") pod \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") "
Jul 15 05:41:18.868057 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.867916    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls\") pod \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") "
Jul 15 05:41:18.868057 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.867955    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3376b29f-aee7-4e7f-a86e-ef9fb338e240-openshift-service-ca\") pod \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\" (UID: \"3376b29f-aee7-4e7f-a86e-ef9fb338e240\") "
Jul 15 05:41:18.868397 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.868363    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3376b29f-aee7-4e7f-a86e-ef9fb338e240-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3376b29f-aee7-4e7f-a86e-ef9fb338e240" (UID: "3376b29f-aee7-4e7f-a86e-ef9fb338e240"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:41:18.870019 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.869992    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3376b29f-aee7-4e7f-a86e-ef9fb338e240" (UID: "3376b29f-aee7-4e7f-a86e-ef9fb338e240"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:41:18.870109 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.870025    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3376b29f-aee7-4e7f-a86e-ef9fb338e240-kube-api-access-ns5z2" (OuterVolumeSpecName: "kube-api-access-ns5z2") pod "3376b29f-aee7-4e7f-a86e-ef9fb338e240" (UID: "3376b29f-aee7-4e7f-a86e-ef9fb338e240"). InnerVolumeSpecName "kube-api-access-ns5z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:41:18.968743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.968723    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3376b29f-aee7-4e7f-a86e-ef9fb338e240-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:18.968743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.968744    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3376b29f-aee7-4e7f-a86e-ef9fb338e240-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:18.968867 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:18.968754    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns5z2\" (UniqueName: \"kubernetes.io/projected/3376b29f-aee7-4e7f-a86e-ef9fb338e240-kube-api-access-ns5z2\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:19.718353 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:19.718327    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"
Jul 15 05:41:19.718801 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:19.718322    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd" event={"ID":"3376b29f-aee7-4e7f-a86e-ef9fb338e240","Type":"ContainerDied","Data":"188096d774dcabe70708f9e5d55598047b01853fc4de6f7e3b33e1644ee62e2a"}
Jul 15 05:41:19.718801 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:19.718490    2581 scope.go:117] "RemoveContainer" containerID="1f2d2f09c8d8b9cc48cf78707b2b959b31cb42eeaee094956d96f43364802788"
Jul 15 05:41:19.739254 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:19.739230    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"]
Jul 15 05:41:19.741097 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:19.741077    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-de675f-9cf4cc74c-xvngd"]
Jul 15 05:41:20.388713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:20.388682    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3376b29f-aee7-4e7f-a86e-ef9fb338e240" path="/var/lib/kubelet/pods/3376b29f-aee7-4e7f-a86e-ef9fb338e240/volumes"
Jul 15 05:41:32.858449 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:32.858407    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"]
Jul 15 05:41:32.859046 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:32.858695    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy" containerID="cri-o://a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23" gracePeriod=30
Jul 15 05:41:33.738220 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738180    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"]
Jul 15 05:41:33.738611 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738597    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy"
Jul 15 05:41:33.738670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738613    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy"
Jul 15 05:41:33.738670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738630    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy"
Jul 15 05:41:33.738670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738635    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy"
Jul 15 05:41:33.738670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738642    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36d3a12f-0ad2-4228-b5e4-c9642e0431fb" containerName="maas-api"
Jul 15 05:41:33.738670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738647    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="36d3a12f-0ad2-4228-b5e4-c9642e0431fb" containerName="maas-api"
Jul 15 05:41:33.738670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738662    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3376b29f-aee7-4e7f-a86e-ef9fb338e240" containerName="maas-api"
Jul 15 05:41:33.738670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738669    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3376b29f-aee7-4e7f-a86e-ef9fb338e240" containerName="maas-api"
Jul 15 05:41:33.739006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738726    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy"
Jul 15 05:41:33.739006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738734    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e509e3-10b4-4cea-904d-fe0dbaf52e03" containerName="istio-proxy"
Jul 15 05:41:33.739006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738743    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="36d3a12f-0ad2-4228-b5e4-c9642e0431fb" containerName="maas-api"
Jul 15 05:41:33.739006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.738749    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3376b29f-aee7-4e7f-a86e-ef9fb338e240" containerName="maas-api"
Jul 15 05:41:33.743847 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.743823    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.746570 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.746539    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ff68b6-openshift-default-dockercfg-d6nbw\""
Jul 15 05:41:33.752145 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.752123    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"]
Jul 15 05:41:33.803634 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-envoy\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.803767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8dnr\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-kube-api-access-w8dnr\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.803767 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803740    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-token\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.803892 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-certs\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.803892 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-data\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.803892 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-socket\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.804021 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istiod-ca-cert\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.804021 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-credential-socket\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.804021 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.803982    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-podinfo\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.904609 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8dnr\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-kube-api-access-w8dnr\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-token\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-certs\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-data\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-socket\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istiod-ca-cert\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-credential-socket\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-podinfo\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.904805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-envoy\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.905139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-certs\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.905139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-socket\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.905260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-credential-socket\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905517 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.905282    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-data\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.905675 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.905540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istiod-ca-cert\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.906926 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.906906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-envoy\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.907244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.907228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-podinfo\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.913857 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.913834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-token\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:33.914047 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:33.914024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8dnr\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-kube-api-access-w8dnr\") pod \"e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") " pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:34.057426 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.057348    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:34.199785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.199758    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"]
Jul 15 05:41:34.201547 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:41:34.201517    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba53313e_30f5_433f_bb40_82f0b11a8ebd.slice/crio-46a644124df4df8e75e229690f5c6cf98ae86644ebce4e3109b39316b793bb79 WatchSource:0}: Error finding container 46a644124df4df8e75e229690f5c6cf98ae86644ebce4e3109b39316b793bb79: Status 404 returned error can't find the container with id 46a644124df4df8e75e229690f5c6cf98ae86644ebce4e3109b39316b793bb79
Jul 15 05:41:34.204047 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.204009    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:34.204130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.204107    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:34.204181 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.204163    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:34.805719 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.805679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" event={"ID":"ba53313e-30f5-433f-bb40-82f0b11a8ebd","Type":"ContainerStarted","Data":"705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1"}
Jul 15 05:41:34.805904 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.805747    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" event={"ID":"ba53313e-30f5-433f-bb40-82f0b11a8ebd","Type":"ContainerStarted","Data":"46a644124df4df8e75e229690f5c6cf98ae86644ebce4e3109b39316b793bb79"}
Jul 15 05:41:34.825555 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:34.825507    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podStartSLOduration=1.8254923349999999 podStartE2EDuration="1.825492335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:41:34.822666385 +0000 UTC m=+2572.990263095" watchObservedRunningTime="2026-07-15 05:41:34.825492335 +0000 UTC m=+2572.993089006"
Jul 15 05:41:35.057584 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:35.057503    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:35.062539 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:35.062516    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:35.810828 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:35.810800    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:35.811689 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:35.811673    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:36.842060 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:36.842031    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"]
Jul 15 05:41:36.852220 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:36.852172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:36.855948 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:36.855588    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ff68b6-serving-cert\""
Jul 15 05:41:36.857149 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:36.857122    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"]
Jul 15 05:41:36.931751 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:36.931714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1148c396-a73b-43b3-bc1f-03718a3ee99a-openshift-service-ca\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:36.931951 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:36.931799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t8hb\" (UniqueName: \"kubernetes.io/projected/1148c396-a73b-43b3-bc1f-03718a3ee99a-kube-api-access-9t8hb\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:36.931951 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:36.931820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.032774 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.032738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t8hb\" (UniqueName: \"kubernetes.io/projected/1148c396-a73b-43b3-bc1f-03718a3ee99a-kube-api-access-9t8hb\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.032956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.032790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.032956 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.032898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1148c396-a73b-43b3-bc1f-03718a3ee99a-openshift-service-ca\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.032956 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:41:37.032943    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-serving-cert: secret "maas-api-e2e-api-a-ff68b6-serving-cert" not found
Jul 15 05:41:37.033131 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:41:37.033015    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls podName:1148c396-a73b-43b3-bc1f-03718a3ee99a nodeName:}" failed. No retries permitted until 2026-07-15 05:41:37.532994068 +0000 UTC m=+2575.700590728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls") pod "maas-api-e2e-api-a-ff68b6-867694f696-wfvrn" (UID: "1148c396-a73b-43b3-bc1f-03718a3ee99a") : secret "maas-api-e2e-api-a-ff68b6-serving-cert" not found
Jul 15 05:41:37.033649 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.033622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1148c396-a73b-43b3-bc1f-03718a3ee99a-openshift-service-ca\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.041765 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.041744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t8hb\" (UniqueName: \"kubernetes.io/projected/1148c396-a73b-43b3-bc1f-03718a3ee99a-kube-api-access-9t8hb\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.537851 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.537810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.540361 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.540333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls\") pod \"maas-api-e2e-api-a-ff68b6-867694f696-wfvrn\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.767256 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.767217    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:37.915330 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.915304    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"]
Jul 15 05:41:37.938029 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:41:37.938003    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1148c396_a73b_43b3_bc1f_03718a3ee99a.slice/crio-4c17bd508599c198fb322c1921374501a5d1475b631785cd646a717e1fbeb19b WatchSource:0}: Error finding container 4c17bd508599c198fb322c1921374501a5d1475b631785cd646a717e1fbeb19b: Status 404 returned error can't find the container with id 4c17bd508599c198fb322c1921374501a5d1475b631785cd646a717e1fbeb19b
Jul 15 05:41:37.963271 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:37.963252    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:41:38.105540 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.105515    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:41:38.245714 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.245684    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4pg8\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-kube-api-access-w4pg8\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.245870 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.245730    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-data\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.245870 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.245770    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-certs\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.245962 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.245940    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-envoy\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.246017 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.245985    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-socket\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.246017 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246001    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:38.246134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835232b-78e6-43a9-9d88-155cd947012a-istiod-ca-cert\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.246134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246124    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835232b-78e6-43a9-9d88-155cd947012a-istio-podinfo\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.246269 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246156    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-istio-token\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.246269 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246191    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-credential-socket\") pod \"e835232b-78e6-43a9-9d88-155cd947012a\" (UID: \"e835232b-78e6-43a9-9d88-155cd947012a\") "
Jul 15 05:41:38.246269 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246197    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:38.246450 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e835232b-78e6-43a9-9d88-155cd947012a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:41:38.246631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246581    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:38.246631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246605    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.246631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246628    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835232b-78e6-43a9-9d88-155cd947012a-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.246898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246641    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.246898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.246736    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-data" (OuterVolumeSpecName: "istio-data") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:38.248090 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.248064    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-kube-api-access-w4pg8" (OuterVolumeSpecName: "kube-api-access-w4pg8") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "kube-api-access-w4pg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:41:38.248301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.248274    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:41:38.248408 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.248317    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-istio-token" (OuterVolumeSpecName: "istio-token") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:41:38.248477 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.248436    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e835232b-78e6-43a9-9d88-155cd947012a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e835232b-78e6-43a9-9d88-155cd947012a" (UID: "e835232b-78e6-43a9-9d88-155cd947012a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:41:38.347732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.347671    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.347732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.347695    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835232b-78e6-43a9-9d88-155cd947012a-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.347732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.347704    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.347732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.347712    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.347732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.347721    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4pg8\" (UniqueName: \"kubernetes.io/projected/e835232b-78e6-43a9-9d88-155cd947012a-kube-api-access-w4pg8\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.347732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.347729    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835232b-78e6-43a9-9d88-155cd947012a-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:41:38.830571 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.830537    2581 generic.go:356] "Generic (PLEG): container finished" podID="e835232b-78e6-43a9-9d88-155cd947012a" containerID="a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23" exitCode=0
Jul 15 05:41:38.830785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.830590    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"
Jul 15 05:41:38.830785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.830619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" event={"ID":"e835232b-78e6-43a9-9d88-155cd947012a","Type":"ContainerDied","Data":"a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23"}
Jul 15 05:41:38.830785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.830668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd" event={"ID":"e835232b-78e6-43a9-9d88-155cd947012a","Type":"ContainerDied","Data":"cd8ce60a2bf4a03a16f8a4466ceaabb71ee5fc5eef5e3cc5d3fd427d9d3ab3a5"}
Jul 15 05:41:38.830785 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.830693    2581 scope.go:117] "RemoveContainer" containerID="a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23"
Jul 15 05:41:38.832535 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.832512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn" event={"ID":"1148c396-a73b-43b3-bc1f-03718a3ee99a","Type":"ContainerStarted","Data":"abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686"}
Jul 15 05:41:38.832670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.832559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn" event={"ID":"1148c396-a73b-43b3-bc1f-03718a3ee99a","Type":"ContainerStarted","Data":"4c17bd508599c198fb322c1921374501a5d1475b631785cd646a717e1fbeb19b"}
Jul 15 05:41:38.832670 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.832628    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:38.840278 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.840262    2581 scope.go:117] "RemoveContainer" containerID="041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e"
Jul 15 05:41:38.848077 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.848063    2581 scope.go:117] "RemoveContainer" containerID="a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23"
Jul 15 05:41:38.848331 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:41:38.848314    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23\": container with ID starting with a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23 not found: ID does not exist" containerID="a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23"
Jul 15 05:41:38.848397 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.848338    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23"} err="failed to get container status \"a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23\": rpc error: code = NotFound desc = could not find container \"a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23\": container with ID starting with a3dd30955b6d9b294213215ed5c26f7cfe8058abb1de73b428716924085cbc23 not found: ID does not exist"
Jul 15 05:41:38.848397 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.848360    2581 scope.go:117] "RemoveContainer" containerID="041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e"
Jul 15 05:41:38.848613 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:41:38.848595    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e\": container with ID starting with 041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e not found: ID does not exist" containerID="041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e"
Jul 15 05:41:38.848667 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.848623    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e"} err="failed to get container status \"041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e\": rpc error: code = NotFound desc = could not find container \"041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e\": container with ID starting with 041f1f92c7717fecd124bf6a7ae14444ef01ca1e2dc0fcbed2a706884784373e not found: ID does not exist"
Jul 15 05:41:38.851106 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.851064    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn" podStartSLOduration=2.333839188 podStartE2EDuration="2.8510518s" totalImagesPullingTime="517.212612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:41:38.849777052 +0000 UTC m=+2577.017373725" watchObservedRunningTime="2026-07-15 05:41:38.8510518 +0000 UTC m=+2577.018648472"
Jul 15 05:41:38.862354 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.862332    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"]
Jul 15 05:41:38.865937 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:38.865913    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-de675f-openshift-default-8679f56945-8p4zd"]
Jul 15 05:41:40.388250 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:40.388222    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e835232b-78e6-43a9-9d88-155cd947012a" path="/var/lib/kubelet/pods/e835232b-78e6-43a9-9d88-155cd947012a/volumes"
Jul 15 05:41:42.044244 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.044214    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"]
Jul 15 05:41:42.045023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.044923    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy"
Jul 15 05:41:42.045023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.044949    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy"
Jul 15 05:41:42.045023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.044963    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy"
Jul 15 05:41:42.045023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.044972    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy"
Jul 15 05:41:42.045354 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.045119    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy"
Jul 15 05:41:42.052152 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.052127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.057187 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.056885    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-870c8a-openshift-default-dockercfg-tr8dx\""
Jul 15 05:41:42.062986 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.062957    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"]
Jul 15 05:41:42.181239 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-credential-socket\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181410 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-certs\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181410 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18027742-2343-4327-9eb7-94948ceaa122-istiod-ca-cert\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181410 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181331    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-envoy\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181410 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18027742-2343-4327-9eb7-94948ceaa122-istio-podinfo\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6str\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-kube-api-access-x6str\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-socket\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181620 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-data\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.181742 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.181630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-istio-token\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282482 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-credential-socket\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282482 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-certs\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18027742-2343-4327-9eb7-94948ceaa122-istiod-ca-cert\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282530    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-envoy\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18027742-2343-4327-9eb7-94948ceaa122-istio-podinfo\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6str\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-kube-api-access-x6str\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-socket\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-data\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.282739 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-istio-token\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.283126 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.282947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-credential-socket\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.283126 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.283095    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-data\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.283235 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.283132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-socket\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.283235 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.283180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-certs\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.283540 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.283516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18027742-2343-4327-9eb7-94948ceaa122-istiod-ca-cert\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.285133 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.285109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-envoy\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.285275 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.285257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18027742-2343-4327-9eb7-94948ceaa122-istio-podinfo\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.291578 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.291553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-istio-token\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.291992 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.291968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6str\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-kube-api-access-x6str\") pod \"e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") " pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.371718 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.371693    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-870c8a-openshift-default-dockercfg-tr8dx\""
Jul 15 05:41:42.379883 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.379851    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:42.516595 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.516566    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"]
Jul 15 05:41:42.518623 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:41:42.518598    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18027742_2343_4327_9eb7_94948ceaa122.slice/crio-8f0e5e50ea703ea1ddbc066f47e9a969813cb58a20f5f5d18b3ccc4717111662 WatchSource:0}: Error finding container 8f0e5e50ea703ea1ddbc066f47e9a969813cb58a20f5f5d18b3ccc4717111662: Status 404 returned error can't find the container with id 8f0e5e50ea703ea1ddbc066f47e9a969813cb58a20f5f5d18b3ccc4717111662
Jul 15 05:41:42.521449 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.521407    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:42.521521 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.521491    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:42.521561 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.521533    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:42.855521 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.855488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" event={"ID":"18027742-2343-4327-9eb7-94948ceaa122","Type":"ContainerStarted","Data":"baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6"}
Jul 15 05:41:42.855691 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.855539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" event={"ID":"18027742-2343-4327-9eb7-94948ceaa122","Type":"ContainerStarted","Data":"8f0e5e50ea703ea1ddbc066f47e9a969813cb58a20f5f5d18b3ccc4717111662"}
Jul 15 05:41:42.874920 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:42.874863    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" podStartSLOduration=0.87484353 podStartE2EDuration="874.84353ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:41:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:41:42.872791701 +0000 UTC m=+2581.040388411" watchObservedRunningTime="2026-07-15 05:41:42.87484353 +0000 UTC m=+2581.042440203"
Jul 15 05:41:43.380691 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:43.380656    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:43.396225 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:43.392467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:43.860515 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:43.860491    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:43.861465 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:43.861447    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:44.844427 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:44.844401    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:41:50.812186 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.812148    2581 patch_prober.go:28] interesting pod/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 15 05:41:50.812632 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.812235    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 15 05:41:50.908597 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.908562    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerID="705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1" exitCode=0
Jul 15 05:41:50.908771 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.908610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" event={"ID":"ba53313e-30f5-433f-bb40-82f0b11a8ebd","Type":"ContainerDied","Data":"705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1"}
Jul 15 05:41:50.909020 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.909006    2581 scope.go:117] "RemoveContainer" containerID="705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1"
Jul 15 05:41:50.910129 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.910099    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:50.910190 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.910170    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:50.910248 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:50.910225    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:51.920355 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:51.920314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" event={"ID":"ba53313e-30f5-433f-bb40-82f0b11a8ebd","Type":"ContainerStarted","Data":"76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23"}
Jul 15 05:41:52.057746 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:52.057701    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:52.057944 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:52.057916    2581 patch_prober.go:28] interesting pod/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 15 05:41:52.057996 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:52.057978    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 15 05:41:53.058074 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:53.058037    2581 patch_prober.go:28] interesting pod/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 15 05:41:53.058571 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:53.058094    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 15 05:41:54.058393 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:54.058360    2581 patch_prober.go:28] interesting pod/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 15 05:41:54.058758 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:54.058412    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 15 05:41:55.058573 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:55.058533    2581 patch_prober.go:28] interesting pod/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Jul 15 05:41:55.059067 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:55.058592    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Jul 15 05:41:56.062606 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:56.062570    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:56.063009 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:56.062979    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:56.063800 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:56.063775    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:41:57.958234 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:57.958133    2581 generic.go:356] "Generic (PLEG): container finished" podID="18027742-2343-4327-9eb7-94948ceaa122" containerID="baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6" exitCode=0
Jul 15 05:41:57.958234 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:57.958168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" event={"ID":"18027742-2343-4327-9eb7-94948ceaa122","Type":"ContainerDied","Data":"baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6"}
Jul 15 05:41:57.958687 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:57.958578    2581 scope.go:117] "RemoveContainer" containerID="baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6"
Jul 15 05:41:57.959579 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:57.959553    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:57.959665 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:57.959619    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:57.959665 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:57.959649    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:41:58.860942 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:58.860909    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:58.965680 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:58.965641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" event={"ID":"18027742-2343-4327-9eb7-94948ceaa122","Type":"ContainerStarted","Data":"807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891"}
Jul 15 05:41:59.380289 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:59.380254    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:41:59.381676 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:59.381644    2581 patch_prober.go:28] interesting pod/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 15 05:41:59.381793 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:41:59.381719    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 15 05:42:00.380532 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:00.380494    2581 patch_prober.go:28] interesting pod/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 15 05:42:00.381006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:00.380572    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 15 05:42:01.381345 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:01.380883    2581 patch_prober.go:28] interesting pod/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 15 05:42:01.381345 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:01.380943    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 15 05:42:02.381096 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:02.381055    2581 patch_prober.go:28] interesting pod/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Jul 15 05:42:02.381326 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:02.381138    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Jul 15 05:42:03.385673 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:03.385639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:42:03.386179 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:03.386014    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:42:03.386737 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:03.386717    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:42:08.281102 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.281008    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"]
Jul 15 05:42:08.282964 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.282925    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn" podUID="1148c396-a73b-43b3-bc1f-03718a3ee99a" containerName="maas-api" containerID="cri-o://abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686" gracePeriod=30
Jul 15 05:42:08.544451 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.544393    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:42:08.653104 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.653066    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1148c396-a73b-43b3-bc1f-03718a3ee99a-openshift-service-ca\") pod \"1148c396-a73b-43b3-bc1f-03718a3ee99a\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") "
Jul 15 05:42:08.653291 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.653157    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t8hb\" (UniqueName: \"kubernetes.io/projected/1148c396-a73b-43b3-bc1f-03718a3ee99a-kube-api-access-9t8hb\") pod \"1148c396-a73b-43b3-bc1f-03718a3ee99a\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") "
Jul 15 05:42:08.653291 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.653233    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls\") pod \"1148c396-a73b-43b3-bc1f-03718a3ee99a\" (UID: \"1148c396-a73b-43b3-bc1f-03718a3ee99a\") "
Jul 15 05:42:08.653451 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.653420    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1148c396-a73b-43b3-bc1f-03718a3ee99a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1148c396-a73b-43b3-bc1f-03718a3ee99a" (UID: "1148c396-a73b-43b3-bc1f-03718a3ee99a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:42:08.655239 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.655198    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1148c396-a73b-43b3-bc1f-03718a3ee99a" (UID: "1148c396-a73b-43b3-bc1f-03718a3ee99a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:42:08.655239 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.655221    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1148c396-a73b-43b3-bc1f-03718a3ee99a-kube-api-access-9t8hb" (OuterVolumeSpecName: "kube-api-access-9t8hb") pod "1148c396-a73b-43b3-bc1f-03718a3ee99a" (UID: "1148c396-a73b-43b3-bc1f-03718a3ee99a"). InnerVolumeSpecName "kube-api-access-9t8hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:42:08.754112 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.754087    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1148c396-a73b-43b3-bc1f-03718a3ee99a-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:08.754112 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.754110    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1148c396-a73b-43b3-bc1f-03718a3ee99a-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:08.754286 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:08.754120    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t8hb\" (UniqueName: \"kubernetes.io/projected/1148c396-a73b-43b3-bc1f-03718a3ee99a-kube-api-access-9t8hb\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:09.025554 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.025523    2581 generic.go:356] "Generic (PLEG): container finished" podID="1148c396-a73b-43b3-bc1f-03718a3ee99a" containerID="abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686" exitCode=0
Jul 15 05:42:09.025713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.025565    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"
Jul 15 05:42:09.025713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.025575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn" event={"ID":"1148c396-a73b-43b3-bc1f-03718a3ee99a","Type":"ContainerDied","Data":"abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686"}
Jul 15 05:42:09.025713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.025617    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn" event={"ID":"1148c396-a73b-43b3-bc1f-03718a3ee99a","Type":"ContainerDied","Data":"4c17bd508599c198fb322c1921374501a5d1475b631785cd646a717e1fbeb19b"}
Jul 15 05:42:09.025713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.025638    2581 scope.go:117] "RemoveContainer" containerID="abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686"
Jul 15 05:42:09.037991 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.037974    2581 scope.go:117] "RemoveContainer" containerID="abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686"
Jul 15 05:42:09.038282 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:42:09.038262    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686\": container with ID starting with abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686 not found: ID does not exist" containerID="abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686"
Jul 15 05:42:09.038339 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.038291    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686"} err="failed to get container status \"abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686\": rpc error: code = NotFound desc = could not find container \"abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686\": container with ID starting with abff99836c8926a540f6bafb2b51355038e6db2a25e463585cb4492741a5f686 not found: ID does not exist"
Jul 15 05:42:09.049553 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.049521    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"]
Jul 15 05:42:09.053120 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:09.053094    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ff68b6-867694f696-wfvrn"]
Jul 15 05:42:10.388892 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:10.388860    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1148c396-a73b-43b3-bc1f-03718a3ee99a" path="/var/lib/kubelet/pods/1148c396-a73b-43b3-bc1f-03718a3ee99a/volumes"
Jul 15 05:42:23.230775 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:23.230741    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"]
Jul 15 05:42:23.231175 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:23.230997    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy" containerID="cri-o://76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23" gracePeriod=30
Jul 15 05:42:26.065224 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:26.065158    2581 patch_prober.go:28] interesting pod/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 05:42:26.065604 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:26.065263    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 05:42:28.488576 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.488549    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:42:28.620661 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-socket\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.620806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-token\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.620806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620704    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-envoy\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.620806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620741    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-podinfo\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.620806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8dnr\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-kube-api-access-w8dnr\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.620806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620785    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-data\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.620806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620801    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istiod-ca-cert\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.621127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-credential-socket\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.621127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620893    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-certs\") pod \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\" (UID: \"ba53313e-30f5-433f-bb40-82f0b11a8ebd\") "
Jul 15 05:42:28.621127 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.620947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:28.621323 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.621143    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:28.621323 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.621192    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:42:28.621323 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.621311    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.621485 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.621327    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.621485 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.621427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:28.621656 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.621621    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:28.623003 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.622982    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-kube-api-access-w8dnr" (OuterVolumeSpecName: "kube-api-access-w8dnr") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "kube-api-access-w8dnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:42:28.623122 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.623107    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:28.623304 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.623282    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:42:28.623367 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.623348    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba53313e-30f5-433f-bb40-82f0b11a8ebd" (UID: "ba53313e-30f5-433f-bb40-82f0b11a8ebd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:42:28.722624 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.722596    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.722624 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.722621    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.722778 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.722634    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8dnr\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-kube-api-access-w8dnr\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.722778 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.722647    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.722778 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.722659    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.722778 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.722672    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba53313e-30f5-433f-bb40-82f0b11a8ebd-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:28.722778 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:28.722683    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba53313e-30f5-433f-bb40-82f0b11a8ebd-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:29.136927 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.136894    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerID="76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23" exitCode=0
Jul 15 05:42:29.137110 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.136946    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"
Jul 15 05:42:29.137110 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.136977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" event={"ID":"ba53313e-30f5-433f-bb40-82f0b11a8ebd","Type":"ContainerDied","Data":"76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23"}
Jul 15 05:42:29.137110 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.137028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n" event={"ID":"ba53313e-30f5-433f-bb40-82f0b11a8ebd","Type":"ContainerDied","Data":"46a644124df4df8e75e229690f5c6cf98ae86644ebce4e3109b39316b793bb79"}
Jul 15 05:42:29.137110 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.137052    2581 scope.go:117] "RemoveContainer" containerID="76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23"
Jul 15 05:42:29.146894 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.146878    2581 scope.go:117] "RemoveContainer" containerID="705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1"
Jul 15 05:42:29.154556 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.154540    2581 scope.go:117] "RemoveContainer" containerID="76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23"
Jul 15 05:42:29.154918 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:42:29.154888    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23\": container with ID starting with 76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23 not found: ID does not exist" containerID="76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23"
Jul 15 05:42:29.155028 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.154930    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23"} err="failed to get container status \"76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23\": rpc error: code = NotFound desc = could not find container \"76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23\": container with ID starting with 76075d26686e7ff27152b525f234952cfc1f3e2caea1b2f4e9b04b140af9ab23 not found: ID does not exist"
Jul 15 05:42:29.155028 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.154953    2581 scope.go:117] "RemoveContainer" containerID="705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1"
Jul 15 05:42:29.155326 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:42:29.155305    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1\": container with ID starting with 705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1 not found: ID does not exist" containerID="705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1"
Jul 15 05:42:29.155399 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.155330    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1"} err="failed to get container status \"705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1\": rpc error: code = NotFound desc = could not find container \"705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1\": container with ID starting with 705ebc6d05efa80a4ba0b6770cdac30cef799d6a9305ee3e69560258255c43e1 not found: ID does not exist"
Jul 15 05:42:29.157524 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.157504    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"]
Jul 15 05:42:29.161144 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:29.161124    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ff68b6-openshift-default-7f78f59d7-4j97n"]
Jul 15 05:42:30.387934 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:30.387902    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" path="/var/lib/kubelet/pods/ba53313e-30f5-433f-bb40-82f0b11a8ebd/volumes"
Jul 15 05:42:49.360905 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:49.360871    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"]
Jul 15 05:42:49.361334 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:49.361135    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy" containerID="cri-o://807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891" gracePeriod=30
Jul 15 05:42:50.240166 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.240136    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"]
Jul 15 05:42:50.240843 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.240819    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy"
Jul 15 05:42:50.241044 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.241010    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy"
Jul 15 05:42:50.241163 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.241086    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1148c396-a73b-43b3-bc1f-03718a3ee99a" containerName="maas-api"
Jul 15 05:42:50.241163 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.241098    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1148c396-a73b-43b3-bc1f-03718a3ee99a" containerName="maas-api"
Jul 15 05:42:50.241306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.241266    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e835232b-78e6-43a9-9d88-155cd947012a" containerName="istio-proxy"
Jul 15 05:42:50.241306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.241282    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1148c396-a73b-43b3-bc1f-03718a3ee99a" containerName="maas-api"
Jul 15 05:42:50.241306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.241297    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy"
Jul 15 05:42:50.241417 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.241307    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy"
Jul 15 05:42:50.243647 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.243630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.247238 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.246392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d43b4e-openshift-default-dockercfg-dm2bj\""
Jul 15 05:42:50.255255 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.255230    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"]
Jul 15 05:42:50.314135 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-envoy\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314135 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5rc8\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-kube-api-access-v5rc8\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314324 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db2d434-1163-491b-9af2-f4a729f497b1-istiod-ca-cert\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314324 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-credential-socket\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314324 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-socket\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314324 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db2d434-1163-491b-9af2-f4a729f497b1-istio-podinfo\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314489 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-data\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314489 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-istio-token\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.314489 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.314474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-certs\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.415915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.415883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-data\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.415915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.415915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-istio-token\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.415944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-certs\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.415963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-envoy\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5rc8\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-kube-api-access-v5rc8\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db2d434-1163-491b-9af2-f4a729f497b1-istiod-ca-cert\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-credential-socket\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-socket\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db2d434-1163-491b-9af2-f4a729f497b1-istio-podinfo\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-certs\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-data\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416891 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-credential-socket\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416891 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-socket\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.416891 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.416842    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db2d434-1163-491b-9af2-f4a729f497b1-istiod-ca-cert\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.418304 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.418282    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-envoy\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.418674 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.418654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db2d434-1163-491b-9af2-f4a729f497b1-istio-podinfo\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.424971 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.424944    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5rc8\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-kube-api-access-v5rc8\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.425182 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.425161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-istio-token\") pod \"e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") " pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.561547 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.561519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:50.699792 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.699760    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"]
Jul 15 05:42:50.701294 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:42:50.701266    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db2d434_1163_491b_9af2_f4a729f497b1.slice/crio-250ac16e26351b9288548284cdcf3d568965490c67efff06b2820bd83ac6724c WatchSource:0}: Error finding container 250ac16e26351b9288548284cdcf3d568965490c67efff06b2820bd83ac6724c: Status 404 returned error can't find the container with id 250ac16e26351b9288548284cdcf3d568965490c67efff06b2820bd83ac6724c
Jul 15 05:42:50.703520 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.703486    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:42:50.703605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.703553    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:42:50.703605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:50.703585    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:42:51.256330 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:51.256287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" event={"ID":"9db2d434-1163-491b-9af2-f4a729f497b1","Type":"ContainerStarted","Data":"d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca"}
Jul 15 05:42:51.256525 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:51.256355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" event={"ID":"9db2d434-1163-491b-9af2-f4a729f497b1","Type":"ContainerStarted","Data":"250ac16e26351b9288548284cdcf3d568965490c67efff06b2820bd83ac6724c"}
Jul 15 05:42:51.281723 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:51.281673    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" podStartSLOduration=1.281657335 podStartE2EDuration="1.281657335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:42:51.280372484 +0000 UTC m=+2649.447969158" watchObservedRunningTime="2026-07-15 05:42:51.281657335 +0000 UTC m=+2649.449254006"
Jul 15 05:42:51.562172 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:51.562143    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:51.566811 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:51.566780    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:52.261361 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:52.261330    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:52.262239 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:52.262198    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:42:53.427320 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.427292    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"]
Jul 15 05:42:53.427743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.427716    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy"
Jul 15 05:42:53.427743 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.427728    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba53313e-30f5-433f-bb40-82f0b11a8ebd" containerName="istio-proxy"
Jul 15 05:42:53.430543 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.430520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.433110 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.433088    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d43b4e-serving-cert\""
Jul 15 05:42:53.439445 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.439320    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"]
Jul 15 05:42:53.548005 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.547978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd97x\" (UniqueName: \"kubernetes.io/projected/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-kube-api-access-nd97x\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.548194 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.548021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.548194 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.548053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.648780 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.648741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd97x\" (UniqueName: \"kubernetes.io/projected/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-kube-api-access-nd97x\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.648978 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.648795    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.648978 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.648845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.648978 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:42:53.648942    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-serving-cert: secret "maas-api-e2e-shared-a-d43b4e-serving-cert" not found
Jul 15 05:42:53.649248 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:42:53.649015    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls podName:d43ed90d-f5b6-4ae7-b5cd-cd13ca041902 nodeName:}" failed. No retries permitted until 2026-07-15 05:42:54.148994349 +0000 UTC m=+2652.316591001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls") pod "maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g" (UID: "d43ed90d-f5b6-4ae7-b5cd-cd13ca041902") : secret "maas-api-e2e-shared-a-d43b4e-serving-cert" not found
Jul 15 05:42:53.649732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.649710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:53.657254 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:53.657235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd97x\" (UniqueName: \"kubernetes.io/projected/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-kube-api-access-nd97x\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:54.154352 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.154319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:54.156677 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.156651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls\") pod \"maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:54.343979 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.343946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:54.514769 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.514738    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"]
Jul 15 05:42:54.517838 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:42:54.517801    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43ed90d_f5b6_4ae7_b5cd_cd13ca041902.slice/crio-d50f23cdb94807c9114b8666ba7949d51ee618515b5bae10538ed132030750d1 WatchSource:0}: Error finding container d50f23cdb94807c9114b8666ba7949d51ee618515b5bae10538ed132030750d1: Status 404 returned error can't find the container with id d50f23cdb94807c9114b8666ba7949d51ee618515b5bae10538ed132030750d1
Jul 15 05:42:54.607377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.607352    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:42:54.659627 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.659594    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-data\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.659787 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.659653    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-istio-token\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.659787 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.659682    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-credential-socket\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.659787 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.659712    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-certs\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.660055 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660006    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:54.660055 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660032    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:54.660282 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660222    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:54.660282 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.659739    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-socket\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.660423 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660353    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18027742-2343-4327-9eb7-94948ceaa122-istio-podinfo\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.660423 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660401    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-data" (OuterVolumeSpecName: "istio-data") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:54.660423 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660417    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-envoy\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.660595 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660492    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6str\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-kube-api-access-x6str\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.660595 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.660572    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18027742-2343-4327-9eb7-94948ceaa122-istiod-ca-cert\") pod \"18027742-2343-4327-9eb7-94948ceaa122\" (UID: \"18027742-2343-4327-9eb7-94948ceaa122\") "
Jul 15 05:42:54.661274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.661033    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.661274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.661056    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.661274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.661071    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.661274 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.661082    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.661548 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.661285    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18027742-2343-4327-9eb7-94948ceaa122-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:42:54.662697 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.662672    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-kube-api-access-x6str" (OuterVolumeSpecName: "kube-api-access-x6str") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "kube-api-access-x6str". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:42:54.662902 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.662885    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:42:54.662964 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.662889    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18027742-2343-4327-9eb7-94948ceaa122-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:42:54.663006 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.662969    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-istio-token" (OuterVolumeSpecName: "istio-token") pod "18027742-2343-4327-9eb7-94948ceaa122" (UID: "18027742-2343-4327-9eb7-94948ceaa122"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:42:54.761702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.761649    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18027742-2343-4327-9eb7-94948ceaa122-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.761702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.761671    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18027742-2343-4327-9eb7-94948ceaa122-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.761702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.761680    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6str\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-kube-api-access-x6str\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.761702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.761688    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18027742-2343-4327-9eb7-94948ceaa122-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:54.761702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:54.761698    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18027742-2343-4327-9eb7-94948ceaa122-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:42:55.279773 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.279738    2581 generic.go:356] "Generic (PLEG): container finished" podID="18027742-2343-4327-9eb7-94948ceaa122" containerID="807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891" exitCode=0
Jul 15 05:42:55.279958 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.279795    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"
Jul 15 05:42:55.279958 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.279816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" event={"ID":"18027742-2343-4327-9eb7-94948ceaa122","Type":"ContainerDied","Data":"807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891"}
Jul 15 05:42:55.279958 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.279907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv" event={"ID":"18027742-2343-4327-9eb7-94948ceaa122","Type":"ContainerDied","Data":"8f0e5e50ea703ea1ddbc066f47e9a969813cb58a20f5f5d18b3ccc4717111662"}
Jul 15 05:42:55.279958 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.279938    2581 scope.go:117] "RemoveContainer" containerID="807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891"
Jul 15 05:42:55.282359 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.282332    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g" event={"ID":"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902","Type":"ContainerStarted","Data":"1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e"}
Jul 15 05:42:55.282485 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.282382    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g" event={"ID":"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902","Type":"ContainerStarted","Data":"d50f23cdb94807c9114b8666ba7949d51ee618515b5bae10538ed132030750d1"}
Jul 15 05:42:55.282485 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.282430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:42:55.289745 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.289726    2581 scope.go:117] "RemoveContainer" containerID="baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6"
Jul 15 05:42:55.297640 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.297620    2581 scope.go:117] "RemoveContainer" containerID="807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891"
Jul 15 05:42:55.297903 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:42:55.297884    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891\": container with ID starting with 807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891 not found: ID does not exist" containerID="807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891"
Jul 15 05:42:55.297952 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.297912    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891"} err="failed to get container status \"807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891\": rpc error: code = NotFound desc = could not find container \"807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891\": container with ID starting with 807a761e30838cd08b07f31e073a333d29e54ba6c1a4f91168018e8364847891 not found: ID does not exist"
Jul 15 05:42:55.297952 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.297929    2581 scope.go:117] "RemoveContainer" containerID="baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6"
Jul 15 05:42:55.298165 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:42:55.298148    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6\": container with ID starting with baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6 not found: ID does not exist" containerID="baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6"
Jul 15 05:42:55.298247 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.298174    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6"} err="failed to get container status \"baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6\": rpc error: code = NotFound desc = could not find container \"baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6\": container with ID starting with baa3fdf09af4fe07f6225db5a4e148bd902c315e4ef615d8a27d405d5c5367c6 not found: ID does not exist"
Jul 15 05:42:55.303120 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.303084    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g" podStartSLOduration=1.91095725 podStartE2EDuration="2.303072844s" totalImagesPullingTime="392.115594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:42:55.300703559 +0000 UTC m=+2653.468300264" watchObservedRunningTime="2026-07-15 05:42:55.303072844 +0000 UTC m=+2653.470669516"
Jul 15 05:42:55.313567 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.313548    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"]
Jul 15 05:42:55.317375 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:55.317357    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-870c8a-openshift-default-78cd7dfdb7-lndhv"]
Jul 15 05:42:56.387684 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:56.387650    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18027742-2343-4327-9eb7-94948ceaa122" path="/var/lib/kubelet/pods/18027742-2343-4327-9eb7-94948ceaa122/volumes"
Jul 15 05:42:58.284624 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.284595    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"]
Jul 15 05:42:58.285064 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.285050    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy"
Jul 15 05:42:58.285108 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.285066    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy"
Jul 15 05:42:58.285108 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.285080    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy"
Jul 15 05:42:58.285108 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.285089    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy"
Jul 15 05:42:58.285219 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.285172    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy"
Jul 15 05:42:58.285219 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.285180    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18027742-2343-4327-9eb7-94948ceaa122" containerName="istio-proxy"
Jul 15 05:42:58.288421 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.288401    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.291078 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.291051    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1edf20-openshift-default-dockercfg-t6f99\""
Jul 15 05:42:58.300573 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.300551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"]
Jul 15 05:42:58.392647 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-podinfo\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.392837 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392696    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-token\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.392837 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wld9\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-kube-api-access-5wld9\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.392837 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-credential-socket\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.392837 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-data\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.393148 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-envoy\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.393148 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istiod-ca-cert\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.393148 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.392958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-certs\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.393148 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.393020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-socket\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.493710 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.493672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-token\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.493710 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.493709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wld9\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-kube-api-access-5wld9\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.493955 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.493743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-credential-socket\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.493955 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.493793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-data\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.493955 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.493821    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-envoy\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.493955 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.493907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istiod-ca-cert\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494179 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.493973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-certs\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494179 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.494057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-socket\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494179 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.494123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-podinfo\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.494283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-data\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.494291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-credential-socket\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.494380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-certs\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494557 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.494498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-socket\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.494700 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.494676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istiod-ca-cert\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.496130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.496100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-envoy\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.496467 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.496448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-podinfo\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.501190 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.501170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-token\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.501598 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.501577    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wld9\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-kube-api-access-5wld9\") pod \"e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") " pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.601893 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.601859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:58.741794 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.741757    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"]
Jul 15 05:42:58.747483 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:42:58.742375    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f20dbe8_1f60_4366_a98d_619c7b798d0b.slice/crio-d4b45e39090cbc3074440dee1feb97ded383e01e8a0a41c65c814d1648f8ae98 WatchSource:0}: Error finding container d4b45e39090cbc3074440dee1feb97ded383e01e8a0a41c65c814d1648f8ae98: Status 404 returned error can't find the container with id d4b45e39090cbc3074440dee1feb97ded383e01e8a0a41c65c814d1648f8ae98
Jul 15 05:42:58.749600 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.749562    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:42:58.749713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.749649    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:42:58.749713 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:58.749684    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:42:59.309513 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:59.309476    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" event={"ID":"2f20dbe8-1f60-4366-a98d-619c7b798d0b","Type":"ContainerStarted","Data":"7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353"}
Jul 15 05:42:59.309903 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:59.309545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" event={"ID":"2f20dbe8-1f60-4366-a98d-619c7b798d0b","Type":"ContainerStarted","Data":"d4b45e39090cbc3074440dee1feb97ded383e01e8a0a41c65c814d1648f8ae98"}
Jul 15 05:42:59.329325 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:59.329281    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" podStartSLOduration=1.329264583 podStartE2EDuration="1.329264583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:42:59.328300853 +0000 UTC m=+2657.495897537" watchObservedRunningTime="2026-07-15 05:42:59.329264583 +0000 UTC m=+2657.496861254"
Jul 15 05:42:59.602541 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:59.602415    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:42:59.608176 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:42:59.608151    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:43:00.314970 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:00.314936    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:43:00.315843 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:00.315827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:43:01.308030 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:01.307995    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:43:06.351474 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:06.351443    2581 generic.go:356] "Generic (PLEG): container finished" podID="9db2d434-1163-491b-9af2-f4a729f497b1" containerID="d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca" exitCode=0
Jul 15 05:43:06.351882 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:06.351479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" event={"ID":"9db2d434-1163-491b-9af2-f4a729f497b1","Type":"ContainerDied","Data":"d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca"}
Jul 15 05:43:06.351882 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:06.351859    2581 scope.go:117] "RemoveContainer" containerID="d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca"
Jul 15 05:43:06.353005 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:06.352971    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:43:06.353103 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:06.353057    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:43:06.353103 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:06.353092    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:43:07.261816 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:07.261784    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:43:07.359007 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:07.358970    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" event={"ID":"9db2d434-1163-491b-9af2-f4a729f497b1","Type":"ContainerStarted","Data":"68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401"}
Jul 15 05:43:07.561974 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:07.561950    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:43:07.562963 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:07.562940    2581 patch_prober.go:28] interesting pod/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 15 05:43:07.563018 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:07.562989    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 15 05:43:08.562137 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:08.562091    2581 patch_prober.go:28] interesting pod/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 15 05:43:08.562560 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:08.562151    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 15 05:43:09.562137 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:09.562089    2581 patch_prober.go:28] interesting pod/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 15 05:43:09.562330 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:09.562234    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 15 05:43:10.562124 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:10.562092    2581 patch_prober.go:28] interesting pod/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 15 05:43:10.562298 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:10.562145    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 15 05:43:11.566373 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:11.566341    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:43:11.566783 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:11.566650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:43:11.567424 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:11.567403    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:43:13.395282 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:13.395191    2581 generic.go:356] "Generic (PLEG): container finished" podID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerID="7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353" exitCode=0
Jul 15 05:43:13.395282 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:13.395232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" event={"ID":"2f20dbe8-1f60-4366-a98d-619c7b798d0b","Type":"ContainerDied","Data":"7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353"}
Jul 15 05:43:13.395830 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:13.395737    2581 scope.go:117] "RemoveContainer" containerID="7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353"
Jul 15 05:43:13.396768 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:13.396740    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:43:13.396829 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:13.396812    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:43:13.396865 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:13.396845    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 15 05:43:14.410140 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:14.410106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" event={"ID":"2f20dbe8-1f60-4366-a98d-619c7b798d0b","Type":"ContainerStarted","Data":"0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d"}
Jul 15 05:43:14.603037 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:14.602990    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:43:14.604428 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:14.604398    2581 patch_prober.go:28] interesting pod/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 15 05:43:14.604574 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:14.604473    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 15 05:43:15.315622 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:15.315573    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:43:15.602639 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:15.602536    2581 patch_prober.go:28] interesting pod/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 15 05:43:15.602639 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:15.602602    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 15 05:43:16.602289 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:16.602251    2581 patch_prober.go:28] interesting pod/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 15 05:43:16.602513 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:16.602323    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 15 05:43:17.603193 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:17.603156    2581 patch_prober.go:28] interesting pod/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 15 05:43:17.603802 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:17.603760    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 15 05:43:18.606400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:18.606371    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:43:18.607305 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:43:18.607285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:45:42.689860 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:45:42.689827    2581 scope.go:117] "RemoveContainer" containerID="1049044714ed5857d18ffd98324bf67d02499ccdd57a344591c128179bb64a09"
Jul 15 05:49:43.228016 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.227984    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:43.231473 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.231454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:43.234021 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.233996    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wjfck\""
Jul 15 05:49:43.236769 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.236749    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:43.333553 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.333522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2xj7\" (UniqueName: \"kubernetes.io/projected/be2d0d4d-248f-4227-a5a1-1e130626ca8c-kube-api-access-w2xj7\") pod \"test-curl-7590-139709924323520\" (UID: \"be2d0d4d-248f-4227-a5a1-1e130626ca8c\") " pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:43.433949 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.433921    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2xj7\" (UniqueName: \"kubernetes.io/projected/be2d0d4d-248f-4227-a5a1-1e130626ca8c-kube-api-access-w2xj7\") pod \"test-curl-7590-139709924323520\" (UID: \"be2d0d4d-248f-4227-a5a1-1e130626ca8c\") " pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:43.441401 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.441370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2xj7\" (UniqueName: \"kubernetes.io/projected/be2d0d4d-248f-4227-a5a1-1e130626ca8c-kube-api-access-w2xj7\") pod \"test-curl-7590-139709924323520\" (UID: \"be2d0d4d-248f-4227-a5a1-1e130626ca8c\") " pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:43.548838 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.548760    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:43.680555 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.680533    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:43.683114 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:49:43.683082    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe2d0d4d_248f_4227_a5a1_1e130626ca8c.slice/crio-a57a4eb8942344ec28f0a20d6f88551bf5ae12960711bbddfae5864c97590fcf WatchSource:0}: Error finding container a57a4eb8942344ec28f0a20d6f88551bf5ae12960711bbddfae5864c97590fcf: Status 404 returned error can't find the container with id a57a4eb8942344ec28f0a20d6f88551bf5ae12960711bbddfae5864c97590fcf
Jul 15 05:49:43.684854 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:43.684824    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:49:44.434277 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:44.434243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709924323520" event={"ID":"be2d0d4d-248f-4227-a5a1-1e130626ca8c","Type":"ContainerStarted","Data":"a57a4eb8942344ec28f0a20d6f88551bf5ae12960711bbddfae5864c97590fcf"}
Jul 15 05:49:45.441112 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:45.441082    2581 generic.go:356] "Generic (PLEG): container finished" podID="be2d0d4d-248f-4227-a5a1-1e130626ca8c" containerID="d4b6b2a2e1ba8cb49964ad4b6d14b193b50be59dc52f7784e3a6e1d00ed5d371" exitCode=0
Jul 15 05:49:45.441481 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:45.441172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709924323520" event={"ID":"be2d0d4d-248f-4227-a5a1-1e130626ca8c","Type":"ContainerDied","Data":"d4b6b2a2e1ba8cb49964ad4b6d14b193b50be59dc52f7784e3a6e1d00ed5d371"}
Jul 15 05:49:46.596415 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:46.596391    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:46.629096 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:46.629073    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7590-139709924323520_be2d0d4d-248f-4227-a5a1-1e130626ca8c/test-curl-7590-139709924323520/0.log"
Jul 15 05:49:46.652249 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:46.652226    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:46.655993 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:46.655973    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:46.762679 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:46.762605    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2xj7\" (UniqueName: \"kubernetes.io/projected/be2d0d4d-248f-4227-a5a1-1e130626ca8c-kube-api-access-w2xj7\") pod \"be2d0d4d-248f-4227-a5a1-1e130626ca8c\" (UID: \"be2d0d4d-248f-4227-a5a1-1e130626ca8c\") "
Jul 15 05:49:46.764657 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:46.764630    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2d0d4d-248f-4227-a5a1-1e130626ca8c-kube-api-access-w2xj7" (OuterVolumeSpecName: "kube-api-access-w2xj7") pod "be2d0d4d-248f-4227-a5a1-1e130626ca8c" (UID: "be2d0d4d-248f-4227-a5a1-1e130626ca8c"). InnerVolumeSpecName "kube-api-access-w2xj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:49:46.863997 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:46.863966    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2xj7\" (UniqueName: \"kubernetes.io/projected/be2d0d4d-248f-4227-a5a1-1e130626ca8c-kube-api-access-w2xj7\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:49:47.452687 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:47.452655    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a57a4eb8942344ec28f0a20d6f88551bf5ae12960711bbddfae5864c97590fcf"
Jul 15 05:49:47.452687 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:47.452661    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:47.462575 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:49:47.462547    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7590-139709924323520\" is forbidden: User \"system:node:ip-10-0-130-29.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-29.ec2.internal' and this object" podUID="be2d0d4d-248f-4227-a5a1-1e130626ca8c" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:48.389528 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:48.389500    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be2d0d4d-248f-4227-a5a1-1e130626ca8c" path="/var/lib/kubelet/pods/be2d0d4d-248f-4227-a5a1-1e130626ca8c/volumes"
Jul 15 05:49:49.193426 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.193387    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:49.193898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.193878    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d0d4d-248f-4227-a5a1-1e130626ca8c" containerName="test-curl-7590-139709924323520"
Jul 15 05:49:49.193993 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.193900    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d0d4d-248f-4227-a5a1-1e130626ca8c" containerName="test-curl-7590-139709924323520"
Jul 15 05:49:49.194052 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.194031    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2d0d4d-248f-4227-a5a1-1e130626ca8c" containerName="test-curl-7590-139709924323520"
Jul 15 05:49:49.197394 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.197372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:49.199879 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.199839    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wjfck\""
Jul 15 05:49:49.202755 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.202730    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:49.285542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.285513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmpmz\" (UniqueName: \"kubernetes.io/projected/39164200-fee6-491c-8387-9115098ec241-kube-api-access-hmpmz\") pod \"test-curl-7590-139709926249792\" (UID: \"39164200-fee6-491c-8387-9115098ec241\") " pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:49.386026 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.385997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmpmz\" (UniqueName: \"kubernetes.io/projected/39164200-fee6-491c-8387-9115098ec241-kube-api-access-hmpmz\") pod \"test-curl-7590-139709926249792\" (UID: \"39164200-fee6-491c-8387-9115098ec241\") " pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:49.393540 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.393516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmpmz\" (UniqueName: \"kubernetes.io/projected/39164200-fee6-491c-8387-9115098ec241-kube-api-access-hmpmz\") pod \"test-curl-7590-139709926249792\" (UID: \"39164200-fee6-491c-8387-9115098ec241\") " pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:49.507829 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.507757    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:49.643977 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:49.643952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:49.645351 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:49:49.645326    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39164200_fee6_491c_8387_9115098ec241.slice/crio-f9427cfb0d2b6807470108186f86a56afb6e45664f0f4ee7082c15991eebbc14 WatchSource:0}: Error finding container f9427cfb0d2b6807470108186f86a56afb6e45664f0f4ee7082c15991eebbc14: Status 404 returned error can't find the container with id f9427cfb0d2b6807470108186f86a56afb6e45664f0f4ee7082c15991eebbc14
Jul 15 05:49:50.468638 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:50.468607    2581 generic.go:356] "Generic (PLEG): container finished" podID="39164200-fee6-491c-8387-9115098ec241" containerID="ec8f9fa301c4b89522188e4656b88f77c5b984232f0af6f1482d1ea726b2e9f7" exitCode=0
Jul 15 05:49:50.468980 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:50.468642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709926249792" event={"ID":"39164200-fee6-491c-8387-9115098ec241","Type":"ContainerDied","Data":"ec8f9fa301c4b89522188e4656b88f77c5b984232f0af6f1482d1ea726b2e9f7"}
Jul 15 05:49:50.468980 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:50.468667    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709926249792" event={"ID":"39164200-fee6-491c-8387-9115098ec241","Type":"ContainerStarted","Data":"f9427cfb0d2b6807470108186f86a56afb6e45664f0f4ee7082c15991eebbc14"}
Jul 15 05:49:51.616130 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:51.616108    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:51.634863 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:51.634845    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7590-139709926249792_39164200-fee6-491c-8387-9115098ec241/test-curl-7590-139709926249792/0.log"
Jul 15 05:49:51.659615 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:51.659589    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:51.663187 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:51.663168    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:51.706770 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:51.706751    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmpmz\" (UniqueName: \"kubernetes.io/projected/39164200-fee6-491c-8387-9115098ec241-kube-api-access-hmpmz\") pod \"39164200-fee6-491c-8387-9115098ec241\" (UID: \"39164200-fee6-491c-8387-9115098ec241\") "
Jul 15 05:49:51.708733 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:51.708710    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39164200-fee6-491c-8387-9115098ec241-kube-api-access-hmpmz" (OuterVolumeSpecName: "kube-api-access-hmpmz") pod "39164200-fee6-491c-8387-9115098ec241" (UID: "39164200-fee6-491c-8387-9115098ec241"). InnerVolumeSpecName "kube-api-access-hmpmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:49:51.808498 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:51.808469    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmpmz\" (UniqueName: \"kubernetes.io/projected/39164200-fee6-491c-8387-9115098ec241-kube-api-access-hmpmz\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:49:52.388695 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:52.388665    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39164200-fee6-491c-8387-9115098ec241" path="/var/lib/kubelet/pods/39164200-fee6-491c-8387-9115098ec241/volumes"
Jul 15 05:49:52.479525 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:52.479500    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:52.479662 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:49:52.479553    2581 scope.go:117] "RemoveContainer" containerID="ec8f9fa301c4b89522188e4656b88f77c5b984232f0af6f1482d1ea726b2e9f7"
Jul 15 05:50:04.679966 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:04.679647    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"]
Jul 15 05:50:04.679966 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:04.679860    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g" podUID="d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" containerName="maas-api" containerID="cri-o://1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e" gracePeriod=30
Jul 15 05:50:05.136759 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.136737    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:50:05.223805 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.223731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd97x\" (UniqueName: \"kubernetes.io/projected/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-kube-api-access-nd97x\") pod \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") "
Jul 15 05:50:05.223805 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.223762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-openshift-service-ca\") pod \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") "
Jul 15 05:50:05.223969 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.223816    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls\") pod \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\" (UID: \"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902\") "
Jul 15 05:50:05.224184 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.224156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" (UID: "d43ed90d-f5b6-4ae7-b5cd-cd13ca041902"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:50:05.225747 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.225719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" (UID: "d43ed90d-f5b6-4ae7-b5cd-cd13ca041902"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:50:05.225854 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.225780    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-kube-api-access-nd97x" (OuterVolumeSpecName: "kube-api-access-nd97x") pod "d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" (UID: "d43ed90d-f5b6-4ae7-b5cd-cd13ca041902"). InnerVolumeSpecName "kube-api-access-nd97x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:50:05.324698 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.324667    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd97x\" (UniqueName: \"kubernetes.io/projected/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-kube-api-access-nd97x\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:05.324698 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.324695    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:05.324698 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.324706    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:05.545766 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.545683    2581 generic.go:356] "Generic (PLEG): container finished" podID="d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" containerID="1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e" exitCode=0
Jul 15 05:50:05.545766 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.545730    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"
Jul 15 05:50:05.545766 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.545737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g" event={"ID":"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902","Type":"ContainerDied","Data":"1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e"}
Jul 15 05:50:05.546046 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.545803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g" event={"ID":"d43ed90d-f5b6-4ae7-b5cd-cd13ca041902","Type":"ContainerDied","Data":"d50f23cdb94807c9114b8666ba7949d51ee618515b5bae10538ed132030750d1"}
Jul 15 05:50:05.546046 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.545835    2581 scope.go:117] "RemoveContainer" containerID="1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e"
Jul 15 05:50:05.560249 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.560196    2581 scope.go:117] "RemoveContainer" containerID="1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e"
Jul 15 05:50:05.560506 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:50:05.560486    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e\": container with ID starting with 1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e not found: ID does not exist" containerID="1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e"
Jul 15 05:50:05.560579 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.560515    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e"} err="failed to get container status \"1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e\": rpc error: code = NotFound desc = could not find container \"1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e\": container with ID starting with 1be41f64a875d722425bb2727425748f258d6c66b02b9e71e601a7fdd9ff822e not found: ID does not exist"
Jul 15 05:50:05.570678 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.570653    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"]
Jul 15 05:50:05.572545 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:05.572526    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d43b4e-b9984bfb5-4c52g"]
Jul 15 05:50:06.388290 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:06.388259    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" path="/var/lib/kubelet/pods/d43ed90d-f5b6-4ae7-b5cd-cd13ca041902/volumes"
Jul 15 05:50:20.073313 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:20.073186    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"]
Jul 15 05:50:20.074458 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:20.074420    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy" containerID="cri-o://68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401" gracePeriod=30
Jul 15 05:50:25.331023 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.330997    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:50:25.405301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405235    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db2d434-1163-491b-9af2-f4a729f497b1-istiod-ca-cert\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405301    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-credential-socket\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405507 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405317    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-istio-token\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405507 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405344    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db2d434-1163-491b-9af2-f4a729f497b1-istio-podinfo\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405507 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-certs\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405507 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405400    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-data\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405507 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405422    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-socket\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405507 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405483    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-envoy\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5rc8\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-kube-api-access-v5rc8\") pod \"9db2d434-1163-491b-9af2-f4a729f497b1\" (UID: \"9db2d434-1163-491b-9af2-f4a729f497b1\") "
Jul 15 05:50:25.405825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405663    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:25.405825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405675    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:25.405825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405685    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db2d434-1163-491b-9af2-f4a729f497b1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:50:25.405825 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405755    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:25.406049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405867    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.406049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405884    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.406049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405897    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.406049 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.405908    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db2d434-1163-491b-9af2-f4a729f497b1-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.406252 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.406221    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-data" (OuterVolumeSpecName: "istio-data") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:25.407643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.407620    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:25.407643 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.407628    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-kube-api-access-v5rc8" (OuterVolumeSpecName: "kube-api-access-v5rc8") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "kube-api-access-v5rc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:50:25.407763 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.407648    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:50:25.407763 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.407725    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9db2d434-1163-491b-9af2-f4a729f497b1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9db2d434-1163-491b-9af2-f4a729f497b1" (UID: "9db2d434-1163-491b-9af2-f4a729f497b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:50:25.507142 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.507121    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.507142 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.507141    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db2d434-1163-491b-9af2-f4a729f497b1-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.507356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.507149    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.507356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.507157    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db2d434-1163-491b-9af2-f4a729f497b1-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.507356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.507165    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5rc8\" (UniqueName: \"kubernetes.io/projected/9db2d434-1163-491b-9af2-f4a729f497b1-kube-api-access-v5rc8\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:25.655126 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.655100    2581 generic.go:356] "Generic (PLEG): container finished" podID="9db2d434-1163-491b-9af2-f4a729f497b1" containerID="68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401" exitCode=0
Jul 15 05:50:25.655271 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.655160    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"
Jul 15 05:50:25.655271 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.655182    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" event={"ID":"9db2d434-1163-491b-9af2-f4a729f497b1","Type":"ContainerDied","Data":"68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401"}
Jul 15 05:50:25.655271 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.655266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr" event={"ID":"9db2d434-1163-491b-9af2-f4a729f497b1","Type":"ContainerDied","Data":"250ac16e26351b9288548284cdcf3d568965490c67efff06b2820bd83ac6724c"}
Jul 15 05:50:25.655422 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.655298    2581 scope.go:117] "RemoveContainer" containerID="68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401"
Jul 15 05:50:25.665362 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.665345    2581 scope.go:117] "RemoveContainer" containerID="d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca"
Jul 15 05:50:25.673608 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.673592    2581 scope.go:117] "RemoveContainer" containerID="68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401"
Jul 15 05:50:25.673855 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:50:25.673837    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401\": container with ID starting with 68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401 not found: ID does not exist" containerID="68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401"
Jul 15 05:50:25.673915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.673859    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401"} err="failed to get container status \"68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401\": rpc error: code = NotFound desc = could not find container \"68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401\": container with ID starting with 68289d29d35825028717d66c42d16a7a434671853254208ef990a5b33b0c7401 not found: ID does not exist"
Jul 15 05:50:25.673915 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.673874    2581 scope.go:117] "RemoveContainer" containerID="d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca"
Jul 15 05:50:25.674107 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:50:25.674086    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca\": container with ID starting with d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca not found: ID does not exist" containerID="d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca"
Jul 15 05:50:25.674147 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.674116    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca"} err="failed to get container status \"d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca\": rpc error: code = NotFound desc = could not find container \"d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca\": container with ID starting with d66ae900ed9de2673d86c0269351f34b968844632fd38f9a7a14c91511ab65ca not found: ID does not exist"
Jul 15 05:50:25.679330 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.679306    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"]
Jul 15 05:50:25.682594 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:25.682575    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d43b4e-openshift-default-7c9bf5547-b2rnr"]
Jul 15 05:50:26.388908 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:26.388874    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" path="/var/lib/kubelet/pods/9db2d434-1163-491b-9af2-f4a729f497b1/volumes"
Jul 15 05:50:46.258541 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:46.258508    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"]
Jul 15 05:50:46.259369 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:46.259335    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy" containerID="cri-o://0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d" gracePeriod=30
Jul 15 05:50:46.942306 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:46.942273    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8645cb74c8-w9k5t_1716f25f-6c15-460f-ae10-bb47ce97579a/authorino/0.log"
Jul 15 05:50:48.609128 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:48.609095    2581 patch_prober.go:28] interesting pod/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 05:50:48.609588 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:48.609147    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 05:50:51.515676 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.515650    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:50:51.525226 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wld9\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-kube-api-access-5wld9\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525226 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525190    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-envoy\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525234    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-token\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-data\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525287    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istiod-ca-cert\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525309    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-socket\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525364    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-certs\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525400 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525392    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-podinfo\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525718 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525427    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-credential-socket\") pod \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\" (UID: \"2f20dbe8-1f60-4366-a98d-619c7b798d0b\") "
Jul 15 05:50:51.525779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525712    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:50:51.526163 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.525956    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:51.526163 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.526107    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:51.526396 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.526161    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:51.526396 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.526255    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:51.527496 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.527471    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:50:51.527605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.527531    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-kube-api-access-5wld9" (OuterVolumeSpecName: "kube-api-access-5wld9") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "kube-api-access-5wld9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:50:51.527605 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.527597    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 05:50:51.527922 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.527895    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f20dbe8-1f60-4366-a98d-619c7b798d0b" (UID: "2f20dbe8-1f60-4366-a98d-619c7b798d0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:50:51.626542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626513    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-credential-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626542 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626539    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wld9\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-kube-api-access-5wld9\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626549    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-envoy\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626558    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626566    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-data\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626574    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istiod-ca-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626582    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-socket\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626590    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f20dbe8-1f60-4366-a98d-619c7b798d0b-workload-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.626749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.626598    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f20dbe8-1f60-4366-a98d-619c7b798d0b-istio-podinfo\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 05:50:51.788625 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.788556    2581 generic.go:356] "Generic (PLEG): container finished" podID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerID="0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d" exitCode=0
Jul 15 05:50:51.788625 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.788607    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"
Jul 15 05:50:51.788776 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.788636    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" event={"ID":"2f20dbe8-1f60-4366-a98d-619c7b798d0b","Type":"ContainerDied","Data":"0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d"}
Jul 15 05:50:51.788776 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.788697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm" event={"ID":"2f20dbe8-1f60-4366-a98d-619c7b798d0b","Type":"ContainerDied","Data":"d4b45e39090cbc3074440dee1feb97ded383e01e8a0a41c65c814d1648f8ae98"}
Jul 15 05:50:51.788776 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.788728    2581 scope.go:117] "RemoveContainer" containerID="0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d"
Jul 15 05:50:51.798138 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.798117    2581 scope.go:117] "RemoveContainer" containerID="7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353"
Jul 15 05:50:51.806005 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.805991    2581 scope.go:117] "RemoveContainer" containerID="0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d"
Jul 15 05:50:51.806250 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:50:51.806228    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d\": container with ID starting with 0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d not found: ID does not exist" containerID="0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d"
Jul 15 05:50:51.806301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.806259    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d"} err="failed to get container status \"0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d\": rpc error: code = NotFound desc = could not find container \"0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d\": container with ID starting with 0be122aa7b7568d09e9419787222e5ffd6132d1d9e13cd25f9852024f36a0c0d not found: ID does not exist"
Jul 15 05:50:51.806301 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.806276    2581 scope.go:117] "RemoveContainer" containerID="7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353"
Jul 15 05:50:51.806559 ip-10-0-130-29 kubenswrapper[2581]: E0715 05:50:51.806533    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353\": container with ID starting with 7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353 not found: ID does not exist" containerID="7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353"
Jul 15 05:50:51.806644 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.806567    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353"} err="failed to get container status \"7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353\": rpc error: code = NotFound desc = could not find container \"7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353\": container with ID starting with 7b429411924b8f0866efeec6494af587800a69af84a862b2ca0a43312f0b2353 not found: ID does not exist"
Jul 15 05:50:51.809103 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.809082    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"]
Jul 15 05:50:51.815348 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.815328    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1edf20-openshift-default-64d696c947-fvkvm"]
Jul 15 05:50:51.961690 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:51.961668    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-92dw4_3b47826b-4ca2-431a-a537-e77aa23881f1/manager/0.log"
Jul 15 05:50:52.090732 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:52.090707    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-g4g9w_25b6a7f1-e50a-4e38-a0db-06739cd7715e/manager/0.log"
Jul 15 05:50:52.220368 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:52.220336    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77f5789774-stv5r_4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8/manager/0.log"
Jul 15 05:50:52.350104 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:52.350026    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dvhc6_2aa0e25e-b678-48cb-b7fb-dec84b6844ce/server/0.log"
Jul 15 05:50:52.388142 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:52.388115    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" path="/var/lib/kubelet/pods/2f20dbe8-1f60-4366-a98d-619c7b798d0b/volumes"
Jul 15 05:50:52.489781 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:52.489759    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-222x5_5fed6480-2f80-42b3-88c1-7fd28c1a4341/manager/0.log"
Jul 15 05:50:52.743390 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:52.743326    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-mqwp6_607f6c72-cd50-402e-93d2-fab7ebb58980/copy-manifests/0.log"
Jul 15 05:50:52.755806 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:52.755786    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-mqwp6_607f6c72-cd50-402e-93d2-fab7ebb58980/manager/0.log"
Jul 15 05:50:53.272923 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:53.272897    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b456cc7c-wdh64_74accdc7-b5f4-4b56-8c46-232ee4dbf183/maas-api/0.log"
Jul 15 05:50:53.391734 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:53.391707    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734875-s5dp6_f852e8c6-9276-4e30-b240-6403daebaaab/cleanup/0.log"
Jul 15 05:50:55.140941 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:55.140911    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-v99mt_0d669fb4-5dc1-4f50-81ea-caaa168d9fca/postgres/0.log"
Jul 15 05:50:56.439240 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:56.439195    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8645cb74c8-w9k5t_1716f25f-6c15-460f-ae10-bb47ce97579a/authorino/0.log"
Jul 15 05:50:56.572264 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:56.572239    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tfvm2_f49a82f8-2896-49b7-999f-9cc93e98f207/manager/0.log"
Jul 15 05:50:56.925315 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:56.925282    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dz279_6fd60195-c7cb-4386-8e2b-2fb929fe41e8/registry-server/0.log"
Jul 15 05:50:57.049836 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:57.049810    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tn8bc_2f564665-b8e5-435f-9f98-e127b8f0725c/manager/0.log"
Jul 15 05:50:57.667844 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:57.667821    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s7h8g_d0b7352d-a5ae-4e48-aa10-919032c86c1f/istio-proxy/0.log"
Jul 15 05:50:58.028960 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:58.028881    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-wndzn_949df993-3e3a-4c6a-9980-a72574ee759d/kube-auth-proxy/0.log"
Jul 15 05:50:58.150631 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:58.150604    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6czc5_2b2078b7-4b31-42c0-b440-ccfe7c196996/istio-proxy/1.log"
Jul 15 05:50:58.271880 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:58.271850    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-vbn7w_7bb2c09a-a467-4797-8be0-facd56b59937/payload-pre-processing/0.log"
Jul 15 05:50:58.388997 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:58.388971    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s9n2p_996fd7c1-5f9d-467e-a260-622a64156bd8/payload-processing/0.log"
Jul 15 05:50:58.985373 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:58.985346    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gb2m8_1da49ef6-2a86-4512-a219-b9dda647a939/main/0.log"
Jul 15 05:50:59.335671 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:50:59.335646    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lxfbq_8f5eb2cc-61d1-4f67-8d58-53a182263b80/main/0.log"
Jul 15 05:51:06.983379 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:06.983304    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tq7nn_6f13133e-0663-4a5a-b4d6-9a4e14609a76/global-pull-secret-syncer/0.log"
Jul 15 05:51:07.108880 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:07.108856    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lcwg8_2941fa73-a703-4c81-b68b-a1c89a7cd65d/konnectivity-agent/0.log"
Jul 15 05:51:07.152925 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:07.152903    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-29.ec2.internal_46ea3c93a970f7700a2a3849415d7a22/haproxy/0.log"
Jul 15 05:51:11.278539 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:11.278505    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8645cb74c8-w9k5t_1716f25f-6c15-460f-ae10-bb47ce97579a/authorino/0.log"
Jul 15 05:51:11.341681 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:11.341652    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tfvm2_f49a82f8-2896-49b7-999f-9cc93e98f207/manager/0.log"
Jul 15 05:51:11.463166 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:11.463141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dz279_6fd60195-c7cb-4386-8e2b-2fb929fe41e8/registry-server/0.log"
Jul 15 05:51:11.564898 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:11.564873    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tn8bc_2f564665-b8e5-435f-9f98-e127b8f0725c/manager/0.log"
Jul 15 05:51:13.770419 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:13.770389    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-58f4f46dfb-dhnq2_ea60aaee-d573-45f7-8896-7f22d42df686/metrics-server/0.log"
Jul 15 05:51:13.899399 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:13.899373    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cpxwp_8995cb25-9a05-4459-a5f9-5a684d8addda/node-exporter/0.log"
Jul 15 05:51:13.919919 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:13.919901    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cpxwp_8995cb25-9a05-4459-a5f9-5a684d8addda/kube-rbac-proxy/0.log"
Jul 15 05:51:13.938647 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:13.938630    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cpxwp_8995cb25-9a05-4459-a5f9-5a684d8addda/init-textfile/0.log"
Jul 15 05:51:14.049971 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.049909    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hx687_635175b9-d7ac-4ea2-aef1-d6e716193e8c/kube-rbac-proxy-main/0.log"
Jul 15 05:51:14.072518 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.072496    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hx687_635175b9-d7ac-4ea2-aef1-d6e716193e8c/kube-rbac-proxy-self/0.log"
Jul 15 05:51:14.093875 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.093857    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hx687_635175b9-d7ac-4ea2-aef1-d6e716193e8c/openshift-state-metrics/0.log"
Jul 15 05:51:14.473020 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.472999    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bdcbcd669-ssmps_ac839800-6548-4332-98d0-0c5220844e48/thanos-query/0.log"
Jul 15 05:51:14.500747 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.500726    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bdcbcd669-ssmps_ac839800-6548-4332-98d0-0c5220844e48/kube-rbac-proxy-web/0.log"
Jul 15 05:51:14.526549 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.526528    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bdcbcd669-ssmps_ac839800-6548-4332-98d0-0c5220844e48/kube-rbac-proxy/0.log"
Jul 15 05:51:14.552079 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.552057    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bdcbcd669-ssmps_ac839800-6548-4332-98d0-0c5220844e48/prom-label-proxy/0.log"
Jul 15 05:51:14.584840 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.584820    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bdcbcd669-ssmps_ac839800-6548-4332-98d0-0c5220844e48/kube-rbac-proxy-rules/0.log"
Jul 15 05:51:14.609392 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:14.609372    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bdcbcd669-ssmps_ac839800-6548-4332-98d0-0c5220844e48/kube-rbac-proxy-metrics/0.log"
Jul 15 05:51:17.226506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.226478    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c9949b5ff-qnhzh_3babc8bd-ac5b-4f00-8c00-193f5784eae6/console/0.log"
Jul 15 05:51:17.810702 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.810668    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"]
Jul 15 05:51:17.811069 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811057    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" containerName="maas-api"
Jul 15 05:51:17.811118 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811070    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" containerName="maas-api"
Jul 15 05:51:17.811118 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811085    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy"
Jul 15 05:51:17.811118 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811091    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy"
Jul 15 05:51:17.811118 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811102    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy"
Jul 15 05:51:17.811118 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811107    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy"
Jul 15 05:51:17.811118 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811113    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy"
Jul 15 05:51:17.811118 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811118    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811128    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811135    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811148    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39164200-fee6-491c-8387-9115098ec241" containerName="test-curl-7590-139709926249792"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811155    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39164200-fee6-491c-8387-9115098ec241" containerName="test-curl-7590-139709926249792"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811222    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="39164200-fee6-491c-8387-9115098ec241" containerName="test-curl-7590-139709926249792"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811231    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43ed90d-f5b6-4ae7-b5cd-cd13ca041902" containerName="maas-api"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811238    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811245    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811251    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db2d434-1163-491b-9af2-f4a729f497b1" containerName="istio-proxy"
Jul 15 05:51:17.811377 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.811257    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f20dbe8-1f60-4366-a98d-619c7b798d0b" containerName="istio-proxy"
Jul 15 05:51:17.814484 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.814469    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:17.816838 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.816816    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bw8rh\"/\"openshift-service-ca.crt\""
Jul 15 05:51:17.816970 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.816817    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bw8rh\"/\"default-dockercfg-b5756\""
Jul 15 05:51:17.818071 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.818051    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bw8rh\"/\"kube-root-ca.crt\""
Jul 15 05:51:17.823499 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.823476    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"]
Jul 15 05:51:17.941481 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.941455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-sys\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:17.941644 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.941493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfd9r\" (UniqueName: \"kubernetes.io/projected/d3c8cb00-366c-4d4a-a6ee-cb2209938914-kube-api-access-xfd9r\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:17.941644 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.941522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-proc\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:17.941644 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.941596    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-lib-modules\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:17.941758 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:17.941647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-podres\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042213 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-sys\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfd9r\" (UniqueName: \"kubernetes.io/projected/d3c8cb00-366c-4d4a-a6ee-cb2209938914-kube-api-access-xfd9r\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-proc\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-sys\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042356 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-lib-modules\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042493 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-podres\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042493 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-proc\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042493 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-lib-modules\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.042493 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.042489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3c8cb00-366c-4d4a-a6ee-cb2209938914-podres\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.049395 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.049366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfd9r\" (UniqueName: \"kubernetes.io/projected/d3c8cb00-366c-4d4a-a6ee-cb2209938914-kube-api-access-xfd9r\") pod \"perf-node-gather-daemonset-zrznl\" (UID: \"d3c8cb00-366c-4d4a-a6ee-cb2209938914\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.125276 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.125197    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.253018 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.252994    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"]
Jul 15 05:51:18.254944 ip-10-0-130-29 kubenswrapper[2581]: W0715 05:51:18.254920    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd3c8cb00_366c_4d4a_a6ee_cb2209938914.slice/crio-6c0bd6ad9f6c72f36849b3481fe55342d1947a19270bb0ff20c7df7b1e69c28e WatchSource:0}: Error finding container 6c0bd6ad9f6c72f36849b3481fe55342d1947a19270bb0ff20c7df7b1e69c28e: Status 404 returned error can't find the container with id 6c0bd6ad9f6c72f36849b3481fe55342d1947a19270bb0ff20c7df7b1e69c28e
Jul 15 05:51:18.928134 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.928102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl" event={"ID":"d3c8cb00-366c-4d4a-a6ee-cb2209938914","Type":"ContainerStarted","Data":"5425bb731762bb2ab68dba66dd2b51a9690ebfff6a5ce5c76fb9f4cd03d47be4"}
Jul 15 05:51:18.928407 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.928158    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:18.928407 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.928175    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl" event={"ID":"d3c8cb00-366c-4d4a-a6ee-cb2209938914","Type":"ContainerStarted","Data":"6c0bd6ad9f6c72f36849b3481fe55342d1947a19270bb0ff20c7df7b1e69c28e"}
Jul 15 05:51:18.945438 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:18.945399    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl" podStartSLOduration=1.94538507 podStartE2EDuration="1.94538507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 05:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:51:18.944475902 +0000 UTC m=+3157.112072574" watchObservedRunningTime="2026-07-15 05:51:18.94538507 +0000 UTC m=+3157.112981741"
Jul 15 05:51:19.149651 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:19.149627    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8d24l_f07ac58a-7667-42a7-9f88-59801841c8f5/dns/0.log"
Jul 15 05:51:19.168701 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:19.168666    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8d24l_f07ac58a-7667-42a7-9f88-59801841c8f5/kube-rbac-proxy/0.log"
Jul 15 05:51:19.307749 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:19.307685    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7ndr7_f9d9807f-4795-4f9c-8ceb-e41e98fdfed0/dns-node-resolver/0.log"
Jul 15 05:51:19.913041 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:19.913017    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n2nlx_321e957c-8792-4fa3-974d-e9352212e364/node-ca/0.log"
Jul 15 05:51:20.804419 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:20.804392    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s7h8g_d0b7352d-a5ae-4e48-aa10-919032c86c1f/istio-proxy/0.log"
Jul 15 05:51:21.054350 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:21.054302    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-wndzn_949df993-3e3a-4c6a-9980-a72574ee759d/kube-auth-proxy/0.log"
Jul 15 05:51:21.079385 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:21.079360    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6czc5_2b2078b7-4b31-42c0-b440-ccfe7c196996/istio-proxy/0.log"
Jul 15 05:51:21.112786 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:21.112755    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6czc5_2b2078b7-4b31-42c0-b440-ccfe7c196996/istio-proxy/1.log"
Jul 15 05:51:21.138597 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:21.138562    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-vbn7w_7bb2c09a-a467-4797-8be0-facd56b59937/payload-pre-processing/0.log"
Jul 15 05:51:21.167003 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:21.166983    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s9n2p_996fd7c1-5f9d-467e-a260-622a64156bd8/payload-processing/0.log"
Jul 15 05:51:21.906042 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:21.906015    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k9765_6567160b-b7af-4b40-ad93-9e84b7338755/serve-healthcheck-canary/0.log"
Jul 15 05:51:22.523779 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:22.523736    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-knln7_3550e420-fe88-4fb6-aee3-3393f8e2ee21/kube-rbac-proxy/0.log"
Jul 15 05:51:22.543448 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:22.543427    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-knln7_3550e420-fe88-4fb6-aee3-3393f8e2ee21/exporter/0.log"
Jul 15 05:51:22.563045 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:22.563014    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-knln7_3550e420-fe88-4fb6-aee3-3393f8e2ee21/extractor/0.log"
Jul 15 05:51:24.719906 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:24.719876    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-92dw4_3b47826b-4ca2-431a-a537-e77aa23881f1/manager/0.log"
Jul 15 05:51:24.862354 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:24.862326    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-g4g9w_25b6a7f1-e50a-4e38-a0db-06739cd7715e/manager/0.log"
Jul 15 05:51:24.943613 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:24.943571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-zrznl"
Jul 15 05:51:24.959055 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:24.959034    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77f5789774-stv5r_4e8ce7c7-e36b-433f-a8ec-6d0d2a0f5ca8/manager/0.log"
Jul 15 05:51:24.977421 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:24.977361    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dvhc6_2aa0e25e-b678-48cb-b7fb-dec84b6844ce/server/0.log"
Jul 15 05:51:25.037884 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:25.037855    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-222x5_5fed6480-2f80-42b3-88c1-7fd28c1a4341/manager/0.log"
Jul 15 05:51:25.153217 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:25.153189    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-mqwp6_607f6c72-cd50-402e-93d2-fab7ebb58980/manager/0.log"
Jul 15 05:51:25.168154 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:25.168136    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-mqwp6_607f6c72-cd50-402e-93d2-fab7ebb58980/copy-manifests/0.log"
Jul 15 05:51:27.593662 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:27.593639    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s7h8g_d0b7352d-a5ae-4e48-aa10-919032c86c1f/istio-proxy/0.log"
Jul 15 05:51:27.829995 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:27.829975    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-wndzn_949df993-3e3a-4c6a-9980-a72574ee759d/kube-auth-proxy/0.log"
Jul 15 05:51:27.848078 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:27.848057    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6czc5_2b2078b7-4b31-42c0-b440-ccfe7c196996/istio-proxy/0.log"
Jul 15 05:51:27.876296 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:27.876276    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6czc5_2b2078b7-4b31-42c0-b440-ccfe7c196996/istio-proxy/1.log"
Jul 15 05:51:27.897694 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:27.897665    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-vbn7w_7bb2c09a-a467-4797-8be0-facd56b59937/payload-pre-processing/0.log"
Jul 15 05:51:27.915075 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:27.915049    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s9n2p_996fd7c1-5f9d-467e-a260-622a64156bd8/payload-processing/0.log"
Jul 15 05:51:29.372691 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:29.372664    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s7h8g_d0b7352d-a5ae-4e48-aa10-919032c86c1f/istio-proxy/0.log"
Jul 15 05:51:29.609339 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:29.609318    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-wndzn_949df993-3e3a-4c6a-9980-a72574ee759d/kube-auth-proxy/0.log"
Jul 15 05:51:29.635965 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:29.635883    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6czc5_2b2078b7-4b31-42c0-b440-ccfe7c196996/istio-proxy/0.log"
Jul 15 05:51:29.663566 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:29.663541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6czc5_2b2078b7-4b31-42c0-b440-ccfe7c196996/istio-proxy/1.log"
Jul 15 05:51:29.685131 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:29.685110    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-vbn7w_7bb2c09a-a467-4797-8be0-facd56b59937/payload-pre-processing/0.log"
Jul 15 05:51:29.705195 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:29.705175    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-s9n2p_996fd7c1-5f9d-467e-a260-622a64156bd8/payload-processing/0.log"
Jul 15 05:51:33.072836 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.072807    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2f9m2_0d613161-cebc-4757-b35a-ddfb39c8501b/kube-multus-additional-cni-plugins/0.log"
Jul 15 05:51:33.094098 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.094074    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2f9m2_0d613161-cebc-4757-b35a-ddfb39c8501b/egress-router-binary-copy/0.log"
Jul 15 05:51:33.112038 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.112018    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2f9m2_0d613161-cebc-4757-b35a-ddfb39c8501b/cni-plugins/0.log"
Jul 15 05:51:33.130273 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.130245    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2f9m2_0d613161-cebc-4757-b35a-ddfb39c8501b/bond-cni-plugin/0.log"
Jul 15 05:51:33.148688 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.148667    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2f9m2_0d613161-cebc-4757-b35a-ddfb39c8501b/routeoverride-cni/0.log"
Jul 15 05:51:33.167370 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.167350    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2f9m2_0d613161-cebc-4757-b35a-ddfb39c8501b/whereabouts-cni-bincopy/0.log"
Jul 15 05:51:33.190866 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.190838    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2f9m2_0d613161-cebc-4757-b35a-ddfb39c8501b/whereabouts-cni/0.log"
Jul 15 05:51:33.521718 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.521645    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k75zq_012027f8-f425-4989-ac09-9f8f03a78849/kube-multus/0.log"
Jul 15 05:51:33.624506 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.624476    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nx4b2_971ca253-982f-4e83-86ad-cba13a9970e0/network-metrics-daemon/0.log"
Jul 15 05:51:33.640705 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:33.640681    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nx4b2_971ca253-982f-4e83-86ad-cba13a9970e0/kube-rbac-proxy/0.log"
Jul 15 05:51:34.983782 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:34.983708    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/ovn-controller/0.log"
Jul 15 05:51:35.000799 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:35.000774    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/ovn-acl-logging/0.log"
Jul 15 05:51:35.026336 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:35.026304    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/kube-rbac-proxy-node/0.log"
Jul 15 05:51:35.047503 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:35.047486    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/kube-rbac-proxy-ovn-metrics/0.log"
Jul 15 05:51:35.062183 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:35.062166    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/northd/0.log"
Jul 15 05:51:35.083114 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:35.083098    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/nbdb/0.log"
Jul 15 05:51:35.103815 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:35.103785    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/sbdb/0.log"
Jul 15 05:51:35.155597 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:35.155563    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-llwcd_216bc077-7039-40ec-b23c-b3737183c581/ovnkube-controller/0.log"
Jul 15 05:51:36.288597 ip-10-0-130-29 kubenswrapper[2581]: I0715 05:51:36.288571    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-65s67_8d1616bb-ada0-4fdb-8b93-1ea20433aa69/network-check-target-container/0.log"