Jul 29 17:51:19.066324 ip-10-0-130-240 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 17:51:19.066340 ip-10-0-130-240 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 17:51:19.066350 ip-10-0-130-240 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 17:51:19.066649 ip-10-0-130-240 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 17:51:29.315218 ip-10-0-130-240 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 17:51:29.315239 ip-10-0-130-240 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6c31c457102449c698a8119288e4d550 --
Jul 29 17:53:29.642578 ip-10-0-130-240 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 17:53:30.018305 ip-10-0-130-240 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:53:30.018305 ip-10-0-130-240 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 17:53:30.018305 ip-10-0-130-240 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:53:30.018305 ip-10-0-130-240 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 17:53:30.018305 ip-10-0-130-240 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:53:30.018998 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.018358    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 17:53:30.024814 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024793    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:53:30.024814 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024812    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:53:30.024814 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024816    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024820    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024823    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024826    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024836    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024839    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024842    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024847    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024850    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024853    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024856    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024859    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024861    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024866    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024870    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024873    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024876    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024879    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024881    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:53:30.024915 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024884    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024886    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024889    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024891    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024894    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024897    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024900    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024903    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024905    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024908    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024911    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024913    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024916    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024919    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024921    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024924    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024927    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024931    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024934    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024936    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:53:30.025396 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024940    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024943    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024946    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024948    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024950    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024953    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024956    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024958    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024960    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024963    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024967    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024969    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024972    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024975    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024977    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024980    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024983    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024985    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024988    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024990    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:53:30.025884 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024993    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024995    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.024997    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025000    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025003    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025005    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025008    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025010    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025013    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025016    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025019    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025021    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025024    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025027    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025030    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025033    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025036    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025038    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025040    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:53:30.026397 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025043    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025045    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025047    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025050    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025054    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025058    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025061    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025064    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025067    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025070    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025072    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025074    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025077    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025079    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025082    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025084    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025087    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025089    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025091    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:53:30.027036 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025828    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025836    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025840    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025843    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025846    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025849    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025852    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025855    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025858    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025861    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025864    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025867    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025870    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025873    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025876    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025878    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025881    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025883    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025886    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:53:30.027681 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025889    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025892    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025894    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025897    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025899    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025902    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025904    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025907    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025909    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025912    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025915    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025917    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025920    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025922    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025925    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025927    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025930    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025933    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025937    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025940    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:53:30.028153 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025943    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025946    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025949    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025951    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025954    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025957    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025961    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025964    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025967    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025970    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025973    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025977    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025981    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025984    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025986    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025989    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025992    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025994    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025997    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:53:30.028661 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.025999    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026002    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026004    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026007    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026009    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026011    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026014    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026016    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026018    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026021    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026023    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026025    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026028    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026031    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026034    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026036    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026039    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026041    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026044    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026047    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:53:30.029130 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026049    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026051    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026054    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026057    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026059    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026062    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026065    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026067    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026070    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026072    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026075    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026077    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026080    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026082    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026085    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026088    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026090    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026092    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026095    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026097    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:53:30.029624 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.026100    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027732    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027746    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027750    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027753    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027756    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027760    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027763    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027766    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027769    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027772    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027775    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027777    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027780    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027783    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027785    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027788    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027790    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027793    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027796    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:53:30.030106 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027798    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027801    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027804    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027806    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027809    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027811    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027814    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027817    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027819    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027823    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027825    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027828    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027830    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027834    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027838    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027841    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027843    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027846    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027849    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:53:30.030613 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027851    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027854    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027856    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027859    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027861    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027864    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027867    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027871    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027875    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027878    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027880    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027883    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027887    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027889    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027892    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027894    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027897    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027899    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027902    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027904    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:53:30.031073 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027906    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027909    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027911    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027914    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027916    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027920    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027925    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027931    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027934    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027937    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027941    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027944    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027946    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027949    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027952    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027954    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027957    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027959    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027962    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:53:30.031653 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027964    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027967    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027969    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027972    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027974    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027976    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027979    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027981    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027984    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027986    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027989    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027991    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027994    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027996    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.027999    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028001    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028004    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028007    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028009    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028012    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:53:30.032122 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028015    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028017    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028094    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028103    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028110    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028114    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028118    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028122    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028127    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028131    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028134    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028138    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028141    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028144    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028148    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028151    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028153    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028171    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028175    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028179    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028184    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028188    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028191    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028194    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 17:53:30.032657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028198    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028201    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028204    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028207    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028210    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028214    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028216    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028219    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028223    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028226    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028230    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028232    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028235    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028238    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028241    2560 flags.go:64] FLAG: --enable-server="true"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028244    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028248    2560 flags.go:64] FLAG: --event-burst="100"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028251    2560 flags.go:64] FLAG: --event-qps="50"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028254    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028257    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028260    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028280    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028288    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028292    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028295    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 29 17:53:30.033241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028298    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028301    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028304    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028307    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028310    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028313    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028316    2560 flags.go:64] FLAG: --feature-gates=""
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028319    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028322    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028325    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028328    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028331    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028334    2560 flags.go:64] FLAG: --help="false"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028337    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028340    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028342    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028346    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028349    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028352    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028354    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028357    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028360    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028362    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 17:53:30.033850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028365    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028368    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028371    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028373    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028376    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028379    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028383    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028386    2560 flags.go:64] FLAG: --lock-file=""
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028389    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028391    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028394    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028400    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028403    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028406    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028409    2560 flags.go:64] FLAG: --logging-format="text"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028412    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028415    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028418    2560 flags.go:64] FLAG: --manifest-url=""
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028420    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028425    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028428    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028432    2560 flags.go:64] FLAG: --max-pods="110"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028435    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028438    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028440    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 17:53:30.034459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028443    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028446    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028448    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028451    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028459    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028462    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028465    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028468    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028471    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028476    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028478    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028482    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028484    2560 flags.go:64] FLAG: --port="10250"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028487    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028491    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c4256080d9b0e84c"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028495    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028497    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028500    2560 flags.go:64] FLAG: --register-node="true"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028503    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028506    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028509    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028512    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028515    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028518    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028521    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 17:53:30.035071 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028524    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028527    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028530    2560 flags.go:64] FLAG: --runonce="false"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028532    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028535    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028538    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028540    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028543    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028546    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028548    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028551    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028554    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028557    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028559    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028562    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028565    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028567    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028570    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028576    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028578    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028581    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028588    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028591    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028593    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028596    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 17:53:30.035711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028599    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028602    2560 flags.go:64] FLAG: --v="2"
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028605    2560 flags.go:64] FLAG: --version="false"
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028609    2560 flags.go:64] FLAG: --vmodule=""
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028614    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.028617    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028746    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028750    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028753    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028756    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028759    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028762    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028764    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028767    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028769    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028772    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028774    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028777    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028779    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028782    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028784    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:53:30.036364 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028787    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028789    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028792    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028794    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028796    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028799    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028801    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028806    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028809    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028811    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028814    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028816    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028818    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028821    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028823    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028826    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028836    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028840    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028842    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028846    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:53:30.036877 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028849    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028851    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028854    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028856    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028859    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028861    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028864    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028866    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028869    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028872    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028875    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028879    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028882    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028884    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028887    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028889    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028893    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028896    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028899    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028906    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:53:30.037384 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028909    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028911    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028914    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028916    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028919    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028922    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028924    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028927    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028930    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028933    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028935    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028938    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028940    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028943    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028945    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028948    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028950    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028953    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028955    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028957    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:53:30.037869 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028960    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028962    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028965    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028967    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028969    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028972    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028974    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028977    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028979    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028981    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028984    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028990    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028992    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028994    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028997    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.028999    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029002    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029004    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029007    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029010    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:53:30.038410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029012    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029015    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029019    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.029023    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.029497    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.036229    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.036247    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.037038    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.037047    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037113    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037118    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037121    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037125    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037128    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:53:30.038889 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037131    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037134    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037137    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037140    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037143    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037145    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037148    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037151    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037153    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037155    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037158    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037161    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037163    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037165    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037168    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037170    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037173    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037175    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037178    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037180    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:53:30.039237 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037184    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037187    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037189    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037191    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037194    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037197    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037199    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037202    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037204    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037207    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037209    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037212    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037214    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037217    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037219    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037222    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037224    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037227    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037229    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037232    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:53:30.039751 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037234    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037236    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037239    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037241    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037244    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037246    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037248    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037251    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037253    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037256    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037258    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037261    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037279    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037285    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037288    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037290    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037295    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037298    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037300    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037303    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:53:30.040252 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037305    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037309    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037313    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037316    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037319    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037321    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037324    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037327    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037329    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037332    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037335    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037338    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037340    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037343    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037346    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037350    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037354    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037357    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:53:30.040748 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037360    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037363    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037366    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037368    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037371    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037374    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037376    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037379    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037382    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037385    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037388    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037391    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037394    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037397    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037399    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:53:30.041191 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037402    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.037407    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037533    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037537    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037540    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037543    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037546    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037549    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037552    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037554    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037557    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037561    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037564    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037566    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037569    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:53:30.041567 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037572    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037575    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037578    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037580    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037583    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037586    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037588    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037591    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037593    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037595    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037598    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037601    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037604    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037607    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037609    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037612    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037614    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037617    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037619    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037622    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:53:30.041943 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037624    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037627    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037629    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037632    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037634    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037637    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037639    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037641    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037644    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037647    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037649    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037652    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037654    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037656    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037659    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037661    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037663    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037666    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037668    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037670    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:53:30.042517 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037673    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037675    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037677    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037680    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037683    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037686    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037688    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037691    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037693    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037695    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037698    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037700    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037703    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037705    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037708    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037710    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037713    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037715    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037719    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:53:30.042992 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037722    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037725    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037728    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037731    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037734    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037737    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037739    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037742    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037744    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037747    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037750    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037752    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037756    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037759    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037762    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037764    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037767    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037771    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037774    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:53:30.043455 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037777    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037779    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037782    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037785    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037787    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037790    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037793    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:30.037796    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.037801    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.037922    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.039893    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.041292    2560 server.go:1019] "Starting client certificate rotation"
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.041420    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:53:30.043928 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.042326    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:53:30.044251 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.043524    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:53:30.044910 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.044895    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:53:30.056032 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.056012    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 29 17:53:30.059159 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.059130    2560 log.go:25] "Validated CRI v1 image API"
Jul 29 17:53:30.060517 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.060500    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 17:53:30.063305 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.063286    2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 baa999f0-6ec2-4a3e-be41-b96233cecfab:/dev/nvme0n1p3 d9b45120-c5ce-4879-aa77-de34f0a76984:/dev/nvme0n1p4]
Jul 29 17:53:30.063359 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.063305    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 17:53:30.069954 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.069833    2560 manager.go:216] Machine: {Timestamp:2026-07-29 17:53:30.068133482 +0000 UTC m=+0.310094511 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3049373 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20e649045ae0ae05a45e464cf3c9c0 SystemUUID:ec20e649-045a-e0ae-05a4-5e464cf3c9c0 BootID:6c31c457-1024-49c6-98a8-119288e4d550 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:e8:ec:a8:73 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:e8:ec:a8:73 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:75:b1:c8:5e:37 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 17:53:30.069954 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.069939    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 17:53:30.070099 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.070028    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 17:53:30.071392 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.071373    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:53:30.073383 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.073357    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 17:53:30.073534 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.073388    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-240.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 17:53:30.074088 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.074078    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 17:53:30.074121 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.074091    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 17:53:30.074121 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.074098    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:53:30.075087 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.075077    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:53:30.076321 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.076311    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 17:53:30.079121 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079109    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:53:30.079228 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079219    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 17:53:30.079261 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079248    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 17:53:30.079261 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079259    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 17:53:30.079335 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079285    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 17:53:30.079335 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079298    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 29 17:53:30.079335 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079312    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 17:53:30.079714 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079683    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:53:30.079714 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.079702    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:53:30.082032 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.082011    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 17:53:30.083432 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.083418    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 17:53:30.083488 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.083448    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 17:53:30.084107 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084097    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 17:53:30.084139 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084115    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 17:53:30.084139 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084123    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 17:53:30.084139 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084129    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 17:53:30.084139 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084137    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 17:53:30.084258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084143    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 17:53:30.084258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084151    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 17:53:30.084258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084156    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 17:53:30.084258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084163    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 17:53:30.084258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084168    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 17:53:30.084258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084176    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 17:53:30.084258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.084182    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 17:53:30.086233 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.086219    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 17:53:30.086233 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.086232    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 17:53:30.089073 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.089055    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-240.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 17:53:30.089628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.089613    2560 server.go:1268] "Started kubelet"
Jul 29 17:53:30.089696 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.089634    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 17:53:30.089696 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.089634    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 17:53:30.089766 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.089690    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 17:53:30.089766 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.089728    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 17:53:30.089848 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.089828    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 17:53:30.090491 ip-10-0-130-240 systemd[1]: Started Kubernetes Kubelet.
Jul 29 17:53:30.091539 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.091526    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 17:53:30.092033 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.092017    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 17:53:30.094823 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.094804    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 17:53:30.096375 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.096361    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rrhj2"
Jul 29 17:53:30.098006 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.097989    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 17:53:30.098091 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098051    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 17:53:30.098091 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.098082    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.098653 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098580    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 17:53:30.098653 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098598    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 17:53:30.098791 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098690    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 17:53:30.098791 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098714    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 17:53:30.098791 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098723    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 17:53:30.098933 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098909    2560 factory.go:55] Registering systemd factory
Jul 29 17:53:30.098933 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.098923    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 29 17:53:30.099244 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.099224    2560 factory.go:153] Registering CRI-O factory
Jul 29 17:53:30.099351 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.099248    2560 factory.go:223] Registration of the crio container factory successfully
Jul 29 17:53:30.099351 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.099315    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 17:53:30.099351 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.099337    2560 factory.go:103] Registering Raw factory
Jul 29 17:53:30.099565 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.099552    2560 manager.go:1196] Started watching for new ooms in manager
Jul 29 17:53:30.102288 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.102252    2560 manager.go:349] Starting recovery of all containers
Jul 29 17:53:30.104580 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.104559    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rrhj2"
Jul 29 17:53:30.104954 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.104924    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 17:53:30.105630 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.104807    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-240.ec2.internal.18c6d43133fa8339  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-240.ec2.internal,UID:ip-10-0-130-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-240.ec2.internal,},FirstTimestamp:2026-07-29 17:53:30.089521977 +0000 UTC m=+0.331483007,LastTimestamp:2026-07-29 17:53:30.089521977 +0000 UTC m=+0.331483007,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-240.ec2.internal,}"
Jul 29 17:53:30.108112 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.108082    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 17:53:30.112215 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.112197    2560 manager.go:354] Recovery completed
Jul 29 17:53:30.113964 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.113828    2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 29 17:53:30.116919 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.116906    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:53:30.118625 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.118611    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:53:30.118686 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.118637    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:53:30.118686 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.118647    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:53:30.119155 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.119143    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 17:53:30.119155 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.119154    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 17:53:30.119281 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.119169    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:53:30.119281 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.119189    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:53:30.119862 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.119848    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:53:30.119907 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.119875    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:53:30.119907 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.119888    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:53:30.120318 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.120208    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-240.ec2.internal.18c6d43135b6912b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-240.ec2.internal,UID:ip-10-0-130-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-240.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-240.ec2.internal,},FirstTimestamp:2026-07-29 17:53:30.118623531 +0000 UTC m=+0.360584559,LastTimestamp:2026-07-29 17:53:30.118623531 +0000 UTC m=+0.360584559,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-240.ec2.internal,}"
Jul 29 17:53:30.120795 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.120781    2560 policy_none.go:49] "None policy: Start"
Jul 29 17:53:30.120875 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.120798    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 17:53:30.120875 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.120823    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 17:53:30.121931 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.121921    2560 policy_none.go:47] "Start"
Jul 29 17:53:30.160593 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.160578    2560 manager.go:337] "Starting Device Plugin manager"
Jul 29 17:53:30.160658 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.160619    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 17:53:30.160658 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.160633    2560 server.go:87] "Starting device plugin registration server"
Jul 29 17:53:30.160922 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.160905    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 17:53:30.161006 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.160922    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 17:53:30.161106 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.161000    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 17:53:30.161106 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.161076    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 17:53:30.161106 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.161082    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 17:53:30.161618 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.161599    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 17:53:30.161710 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.161644    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.261798 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.261762    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:53:30.263125 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.263109    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:53:30.263204 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.263139    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:53:30.263204 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.263149    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:53:30.263204 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.263176    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.271673 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.271628    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.271673 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.271651    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-240.ec2.internal\": node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.273804 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.273780    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 17:53:30.275392 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.275371    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 17:53:30.275485 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.275409    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 17:53:30.275485 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.275433    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 17:53:30.275485 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.275465    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 17:53:30.277778 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.277758    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:53:30.288847 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.288822    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.376841 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.376141    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal"]
Jul 29 17:53:30.376993 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.376889    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:53:30.377896 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.377882    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:53:30.377970 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.377914    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:53:30.377970 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.377927    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:53:30.379308 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.379294    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:53:30.379422 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.379406    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.379477 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.379441    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:53:30.380818 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.380799    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:53:30.380938 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.380824    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:53:30.380938 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.380842    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:53:30.380938 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.380799    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:53:30.380938 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.380902    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:53:30.380938 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.380916    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:53:30.382108 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.382092    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.382179 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.382115    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:53:30.382752 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.382737    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:53:30.382822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.382767    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:53:30.382822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.382783    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:53:30.389185 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.389170    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.400511 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.400492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff29b82ad41d8bf0cc74890e7834231a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal\" (UID: \"ff29b82ad41d8bf0cc74890e7834231a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.400575 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.400519    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff29b82ad41d8bf0cc74890e7834231a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal\" (UID: \"ff29b82ad41d8bf0cc74890e7834231a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.400575 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.400535    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a82b4de24054f43493689057761ee11c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-240.ec2.internal\" (UID: \"a82b4de24054f43493689057761ee11c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.411015 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.410993    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-240.ec2.internal\" not found" node="ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.415375 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.415360    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-240.ec2.internal\" not found" node="ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.490023 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.489978    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.501386 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.501362    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff29b82ad41d8bf0cc74890e7834231a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal\" (UID: \"ff29b82ad41d8bf0cc74890e7834231a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.501490 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.501394    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a82b4de24054f43493689057761ee11c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-240.ec2.internal\" (UID: \"a82b4de24054f43493689057761ee11c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.501490 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.501411    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff29b82ad41d8bf0cc74890e7834231a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal\" (UID: \"ff29b82ad41d8bf0cc74890e7834231a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.501490 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.501452    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff29b82ad41d8bf0cc74890e7834231a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal\" (UID: \"ff29b82ad41d8bf0cc74890e7834231a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.501490 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.501457    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff29b82ad41d8bf0cc74890e7834231a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal\" (UID: \"ff29b82ad41d8bf0cc74890e7834231a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.501490 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.501477    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a82b4de24054f43493689057761ee11c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-240.ec2.internal\" (UID: \"a82b4de24054f43493689057761ee11c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.590886 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.590806    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.691351 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.691316    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.714860 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.714837    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.719111 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:30.719015    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:30.791962 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.791910    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.892655 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.892562    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:30.992839 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:30.992794    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:31.043038 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.043000    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 17:53:31.043529 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.043164    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 17:53:31.093030 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:31.092998    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:31.113568 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.113531    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 17:48:30 +0000 UTC" deadline="2028-03-29 20:39:12.697940374 +0000 UTC"
Jul 29 17:53:31.113568 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.113564    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14618h45m41.584380289s"
Jul 29 17:53:31.195029 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:31.194933    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-240.ec2.internal\" not found"
Jul 29 17:53:31.201637 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.201615    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:53:31.298731 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.298701    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:31.317515 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.317478    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 17:53:31.320158 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.320138    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal"
Jul 29 17:53:31.328728 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.328709    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 17:53:31.334968 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:31.334933    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff29b82ad41d8bf0cc74890e7834231a.slice/crio-c789cfe22785a6b9ab7c6240c4887809126b5979234a710bd5ae7a6d60dd9b9c WatchSource:0}: Error finding container c789cfe22785a6b9ab7c6240c4887809126b5979234a710bd5ae7a6d60dd9b9c: Status 404 returned error can't find the container with id c789cfe22785a6b9ab7c6240c4887809126b5979234a710bd5ae7a6d60dd9b9c
Jul 29 17:53:31.340333 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.340317    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:53:31.361360 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:31.361325    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82b4de24054f43493689057761ee11c.slice/crio-fb11836abfdd350338e3e5a41800065cbd3d78e41fa6115f97a2bca1abc19ca2 WatchSource:0}: Error finding container fb11836abfdd350338e3e5a41800065cbd3d78e41fa6115f97a2bca1abc19ca2: Status 404 returned error can't find the container with id fb11836abfdd350338e3e5a41800065cbd3d78e41fa6115f97a2bca1abc19ca2
Jul 29 17:53:31.374042 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.374012    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:53:31.594396 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:31.594286    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:53:32.082638 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.082552    2560 apiserver.go:52] "Watching apiserver"
Jul 29 17:53:32.088069 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.088038    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 17:53:32.090776 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.090742    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-g9f6g","openshift-multus/multus-additional-cni-plugins-tkbq8","openshift-multus/network-metrics-daemon-kqpsv","openshift-network-diagnostics/network-check-target-k9wdg","kube-system/global-pull-secret-syncer-bphzv","kube-system/konnectivity-agent-ksc6p","kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rkcjz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal","openshift-multus/multus-vbghm","openshift-network-operator/iptables-alerter-tsnm5","openshift-ovn-kubernetes/ovnkube-node-xw7vr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2","openshift-dns/node-resolver-tlm66"]
Jul 29 17:53:32.092775 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.092750    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.094496 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.094393    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.094773 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.094750    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 17:53:32.095225 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.095208    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 17:53:32.095367 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.095222    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 17:53:32.095605 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.095587    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 17:53:32.095802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.095784    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:32.095884 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.095854    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:32.098223 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.098205    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zqpnk\""
Jul 29 17:53:32.100694 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.100469    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.100694 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.100546    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:32.100849 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.100749    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 17:53:32.100914 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.100871    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 17:53:32.100914 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.100890    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z5hkw\""
Jul 29 17:53:32.101040 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.101015    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 17:53:32.101147 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.101121    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 17:53:32.101263 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.101169    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:32.101263 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.101255    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:32.103700 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.103676    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.105802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.105783    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.107130 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.107113    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.108909 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.108887    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 17:53:32.109044 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109017    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 17:53:32.109139 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109092    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gl8f5\""
Jul 29 17:53:32.109285 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109269    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.109405 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109287    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 17:53:32.109476 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109463    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nlzcj\""
Jul 29 17:53:32.109698 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109680    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:53:32.109779 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109681    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 17:53:32.109828 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109796    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 17:53:32.109915 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.109899    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-424rr\""
Jul 29 17:53:32.110029 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.110010    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.110850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.110823    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-etc-kubernetes\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.110948 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.110868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxk56\" (UniqueName: \"kubernetes.io/projected/44570463-5430-45f8-9cdb-c8ea4fa2927a-kube-api-access-gxk56\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.110948 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.110903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-socket-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.110948 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.110930    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-registration-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.111109 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.110956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.111109 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.110983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44570463-5430-45f8-9cdb-c8ea4fa2927a-cni-binary-copy\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111109 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-conf-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111109 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111030    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.111109 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df55af57-df49-40aa-9e88-a6f8e1a604cf-kubelet-config\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.111109 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111080    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-system-cni-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111109 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111105    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-kubelet\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111128    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-hostroot\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-cni-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111178    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-k8s-cni-cncf-io\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-multus-certs\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111228    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111254    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df55af57-df49-40aa-9e88-a6f8e1a604cf-dbus\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-sys-fs\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-cnibin\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-socket-dir-parent\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111418    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-cni-bin\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111481    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-cni-multus\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.111577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111582    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6r5s\" (UniqueName: \"kubernetes.io/projected/3724628d-c3ad-47a9-be69-337160e0fe78-kube-api-access-n6r5s\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:32.112487 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111624    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-device-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.112874 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.112856    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 17:53:32.113156 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.113136    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.113283 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.113263    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 17:53:32.113586 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.113566    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.113914 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.113894    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xd9ws\""
Jul 29 17:53:32.113970 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.113922    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cfgjb\""
Jul 29 17:53:32.114384 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.111651    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-daemon-config\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.114467 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114420    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:53:32.114467 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114418    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4mh7\" (UniqueName: \"kubernetes.io/projected/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-kube-api-access-c4mh7\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.114578 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114475    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 17:53:32.114578 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-os-release\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.114578 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114523    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 17:53:32.114680 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-netns\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.114718 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114700    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 17:53:32.114772 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.114762    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 17:53:32.115771 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.115752    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9z6nk\""
Jul 29 17:53:32.115972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.115760    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:53:32.116044 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.115972    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 17:53:32.116922 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.116891    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 17:53:32.117230 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.117209    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f9pwc\""
Jul 29 17:53:32.117860 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.117842    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 17:53:32.118200 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.118168    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 17:53:32.118455 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.118429    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 17:53:32.118567 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.118532    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 17:53:32.118770 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.118749    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 17:53:32.152017 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.151985    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v2tpz"
Jul 29 17:53:32.160696 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.160660    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v2tpz"
Jul 29 17:53:32.200725 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.200691    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 17:53:32.215413 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-sys-fs\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.215550 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:32.215550 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-socket-dir-parent\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215550 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215472    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-cni-bin\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215550 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215489    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-sys-fs\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.215550 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215496    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-cni-multus\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215554    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-cni-bin\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215552    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6r5s\" (UniqueName: \"kubernetes.io/projected/3724628d-c3ad-47a9-be69-337160e0fe78-kube-api-access-n6r5s\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215598    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysconfig\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.215619    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215626    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db7r8\" (UniqueName: \"kubernetes.io/projected/43f85418-dbf7-4a41-8ddf-cb461995157e-kube-api-access-db7r8\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215651    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/25a9391d-00df-4c40-80cc-137625065288-serviceca\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215683    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-device-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.215711 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.215699    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs podName:3724628d-c3ad-47a9-be69-337160e0fe78 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:32.715678116 +0000 UTC m=+2.957639154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs") pod "network-metrics-daemon-kqpsv" (UID: "3724628d-c3ad-47a9-be69-337160e0fe78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-run-netns\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215748    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-device-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-daemon-config\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-systemd\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215832    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-cni-multus\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-systemd\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-socket-dir-parent\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215899    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-os-release\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.215951 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215923    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-netns\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.215980    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/073dff5b-1210-4a03-8917-cad6c1e91f46-iptables-alerter-script\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216008    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-kubernetes\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216037    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44570463-5430-45f8-9cdb-c8ea4fa2927a-cni-binary-copy\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216043    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-netns\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-conf-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216104    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216132    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-systemd-units\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216143    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-os-release\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216157    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-etc-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216181    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-sys\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-conf-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216224    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-system-cni-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216247    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbaf110-7713-4aed-b3ad-78f77508ceb5-tmp-dir\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216269    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216298    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216296    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysctl-conf\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216377    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-host\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-system-cni-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-os-release\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216433    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-cni-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216445    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-daemon-config\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-k8s-cni-cncf-io\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216531    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216563    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-k8s-cni-cncf-io\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df55af57-df49-40aa-9e88-a6f8e1a604cf-dbus\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216615    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-multus-cni-dir\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216649    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysctl-d\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216681    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99j6x\" (UniqueName: \"kubernetes.io/projected/2dc05f89-a229-4b2b-9d29-4d3e2521b691-kube-api-access-99j6x\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.216851 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216688    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df55af57-df49-40aa-9e88-a6f8e1a604cf-dbus\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216746    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-lib-modules\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cnibin\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25a9391d-00df-4c40-80cc-137625065288-host\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216917    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-cnibin\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44570463-5430-45f8-9cdb-c8ea4fa2927a-cni-binary-copy\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-cnibin\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.216995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217036    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhpcc\" (UniqueName: \"kubernetes.io/projected/073dff5b-1210-4a03-8917-cad6c1e91f46-kube-api-access-nhpcc\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-ovn\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-env-overrides\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovnkube-script-lib\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217197    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecbaf110-7713-4aed-b3ad-78f77508ceb5-hosts-file\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-cni-netd\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12-konnectivity-ca\") pod \"konnectivity-agent-ksc6p\" (UID: \"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12\") " pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.217719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217294    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-tuned\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217319    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43f85418-dbf7-4a41-8ddf-cb461995157e-tmp\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-system-cni-dir\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4mh7\" (UniqueName: \"kubernetes.io/projected/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-kube-api-access-c4mh7\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5x9b\" (UniqueName: \"kubernetes.io/projected/ecbaf110-7713-4aed-b3ad-78f77508ceb5-kube-api-access-k5x9b\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-node-log\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217486    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-log-socket\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cni-binary-copy\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-etc-kubernetes\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217593    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxk56\" (UniqueName: \"kubernetes.io/projected/44570463-5430-45f8-9cdb-c8ea4fa2927a-kube-api-access-gxk56\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-socket-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-registration-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217661    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df55af57-df49-40aa-9e88-a6f8e1a604cf-kubelet-config\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-kubelet\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217735    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12-agent-certs\") pod \"konnectivity-agent-ksc6p\" (UID: \"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12\") " pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217756    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-socket-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.218555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-modprobe-d\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217786    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df55af57-df49-40aa-9e88-a6f8e1a604cf-kubelet-config\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217819    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217850    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-kubelet\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-registration-dir\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217883    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-hostroot\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/073dff5b-1210-4a03-8917-cad6c1e91f46-host-slash\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.217927    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217936    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-slash\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217945    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-var-lib-kubelet\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-var-lib-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.217989    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret podName:df55af57-df49-40aa-9e88-a6f8e1a604cf nodeName:}" failed. No retries permitted until 2026-07-29 17:53:32.717971421 +0000 UTC m=+2.959932451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret") pod "global-pull-secret-syncer-bphzv" (UID: "df55af57-df49-40aa-9e88-a6f8e1a604cf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217993    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-hostroot\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.217856    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-etc-kubernetes\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-cni-bin\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovnkube-config\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.219409 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218074    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcxfp\" (UniqueName: \"kubernetes.io/projected/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-kube-api-access-jcxfp\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.220240 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-multus-certs\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.220240 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218141    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-run-ovn-kubernetes\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.220240 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44570463-5430-45f8-9cdb-c8ea4fa2927a-host-run-multus-certs\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.220240 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovn-node-metrics-cert\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.220240 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-run\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.220240 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218306    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-var-lib-kubelet\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.220240 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.218333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tctr\" (UniqueName: \"kubernetes.io/projected/25a9391d-00df-4c40-80cc-137625065288-kube-api-access-8tctr\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.225301 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.225271    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 17:53:32.227493 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.227469    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:53:32.227493 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.227494    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:53:32.227672 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.227504    2560 projected.go:196] Error preparing data for projected volume kube-api-access-wckcp for pod openshift-network-diagnostics/network-check-target-k9wdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:32.227672 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.227589    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp podName:d494b6bf-3e7d-4273-90bc-80b156c5a16e nodeName:}" failed. No retries permitted until 2026-07-29 17:53:32.727575789 +0000 UTC m=+2.969536806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wckcp" (UniqueName: "kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp") pod "network-check-target-k9wdg" (UID: "d494b6bf-3e7d-4273-90bc-80b156c5a16e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:32.230111 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.230082    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6r5s\" (UniqueName: \"kubernetes.io/projected/3724628d-c3ad-47a9-be69-337160e0fe78-kube-api-access-n6r5s\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:32.231245 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.231196    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4mh7\" (UniqueName: \"kubernetes.io/projected/1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd-kube-api-access-c4mh7\") pod \"aws-ebs-csi-driver-node-gqxl2\" (UID: \"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.233715 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.233692    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxk56\" (UniqueName: \"kubernetes.io/projected/44570463-5430-45f8-9cdb-c8ea4fa2927a-kube-api-access-gxk56\") pod \"multus-vbghm\" (UID: \"44570463-5430-45f8-9cdb-c8ea4fa2927a\") " pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.280149 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.280089    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal" event={"ID":"a82b4de24054f43493689057761ee11c","Type":"ContainerStarted","Data":"fb11836abfdd350338e3e5a41800065cbd3d78e41fa6115f97a2bca1abc19ca2"}
Jul 29 17:53:32.281376 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.281349    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal" event={"ID":"ff29b82ad41d8bf0cc74890e7834231a","Type":"ContainerStarted","Data":"c789cfe22785a6b9ab7c6240c4887809126b5979234a710bd5ae7a6d60dd9b9c"}
Jul 29 17:53:32.319547 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319509    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysconfig\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.319547 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319554    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db7r8\" (UniqueName: \"kubernetes.io/projected/43f85418-dbf7-4a41-8ddf-cb461995157e-kube-api-access-db7r8\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/25a9391d-00df-4c40-80cc-137625065288-serviceca\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319606    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-run-netns\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-systemd\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysconfig\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-systemd\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319712    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319724    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/073dff5b-1210-4a03-8917-cad6c1e91f46-iptables-alerter-script\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-kubernetes\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-systemd-units\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-etc-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.319846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319821    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-sys\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319867    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbaf110-7713-4aed-b3ad-78f77508ceb5-tmp-dir\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319894    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysctl-conf\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-host\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.319969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-os-release\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320032    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysctl-d\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99j6x\" (UniqueName: \"kubernetes.io/projected/2dc05f89-a229-4b2b-9d29-4d3e2521b691-kube-api-access-99j6x\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-lib-modules\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cnibin\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320176    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25a9391d-00df-4c40-80cc-137625065288-host\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320205    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhpcc\" (UniqueName: \"kubernetes.io/projected/073dff5b-1210-4a03-8917-cad6c1e91f46-kube-api-access-nhpcc\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.320235 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320230    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-ovn\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-env-overrides\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovnkube-script-lib\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320375    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecbaf110-7713-4aed-b3ad-78f77508ceb5-hosts-file\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320399    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-cni-netd\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12-konnectivity-ca\") pod \"konnectivity-agent-ksc6p\" (UID: \"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12\") " pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320447    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-tuned\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320467    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43f85418-dbf7-4a41-8ddf-cb461995157e-tmp\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-system-cni-dir\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320527    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5x9b\" (UniqueName: \"kubernetes.io/projected/ecbaf110-7713-4aed-b3ad-78f77508ceb5-kube-api-access-k5x9b\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320536    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/25a9391d-00df-4c40-80cc-137625065288-serviceca\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-node-log\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320596    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-log-socket\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320611    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-run-netns\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320623    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cni-binary-copy\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320661    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-systemd\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.320807 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-kubelet\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12-agent-certs\") pod \"konnectivity-agent-ksc6p\" (UID: \"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12\") " pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320732    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-modprobe-d\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320755    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320781    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/073dff5b-1210-4a03-8917-cad6c1e91f46-host-slash\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-slash\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320852    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-var-lib-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-cni-bin\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320905    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovnkube-config\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320930    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcxfp\" (UniqueName: \"kubernetes.io/projected/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-kube-api-access-jcxfp\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320959    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-run-ovn-kubernetes\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.320984    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovn-node-metrics-cert\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-run\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-var-lib-kubelet\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecbaf110-7713-4aed-b3ad-78f77508ceb5-hosts-file\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tctr\" (UniqueName: \"kubernetes.io/projected/25a9391d-00df-4c40-80cc-137625065288-kube-api-access-8tctr\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.321676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321101    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-systemd\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321423    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-cni-netd\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/073dff5b-1210-4a03-8917-cad6c1e91f46-iptables-alerter-script\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321747    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-kubernetes\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321795    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-systemd-units\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-etc-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321898    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-sys\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.321947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12-konnectivity-ca\") pod \"konnectivity-agent-ksc6p\" (UID: \"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12\") " pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.322157    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecbaf110-7713-4aed-b3ad-78f77508ceb5-tmp-dir\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.322231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.322391    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysctl-conf\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.322448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-host\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.322569 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.322509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-os-release\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.323145 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.322632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-sysctl-d\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.323145 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.322953    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-lib-modules\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.323145 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.323007    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cnibin\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.323145 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.323048    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25a9391d-00df-4c40-80cc-137625065288-host\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.323301 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.323283    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-run-ovn\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.323770 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.323748    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-env-overrides\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.324277 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.324258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovnkube-script-lib\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.324819 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.324796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.325477 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325452    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-log-socket\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.325706 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325678    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-node-log\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.325790 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325754    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-kubelet\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.325790 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325761    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.325918 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325866    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-modprobe-d\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.325978 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325920    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-system-cni-dir\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.325978 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325929    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2dc05f89-a229-4b2b-9d29-4d3e2521b691-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.325978 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.325974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-slash\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.326152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326024    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-run-ovn-kubernetes\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.326152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326062    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43f85418-dbf7-4a41-8ddf-cb461995157e-etc-tuned\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.326152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326065    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/073dff5b-1210-4a03-8917-cad6c1e91f46-host-slash\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.326152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2dc05f89-a229-4b2b-9d29-4d3e2521b691-cni-binary-copy\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.326152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326101    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-run\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.326152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326138    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-host-cni-bin\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.326152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-var-lib-openvswitch\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.326566 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326153    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43f85418-dbf7-4a41-8ddf-cb461995157e-var-lib-kubelet\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.326877 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.326850    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovnkube-config\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.328251 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.328229    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43f85418-dbf7-4a41-8ddf-cb461995157e-tmp\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.330102 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.330080    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12-agent-certs\") pod \"konnectivity-agent-ksc6p\" (UID: \"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12\") " pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.330209 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.330116    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tctr\" (UniqueName: \"kubernetes.io/projected/25a9391d-00df-4c40-80cc-137625065288-kube-api-access-8tctr\") pod \"node-ca-g9f6g\" (UID: \"25a9391d-00df-4c40-80cc-137625065288\") " pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.331146 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.331093    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db7r8\" (UniqueName: \"kubernetes.io/projected/43f85418-dbf7-4a41-8ddf-cb461995157e-kube-api-access-db7r8\") pod \"tuned-rkcjz\" (UID: \"43f85418-dbf7-4a41-8ddf-cb461995157e\") " pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.331920 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.331899    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-ovn-node-metrics-cert\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.334302 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.333917    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhpcc\" (UniqueName: \"kubernetes.io/projected/073dff5b-1210-4a03-8917-cad6c1e91f46-kube-api-access-nhpcc\") pod \"iptables-alerter-tsnm5\" (UID: \"073dff5b-1210-4a03-8917-cad6c1e91f46\") " pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.335141 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.335100    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99j6x\" (UniqueName: \"kubernetes.io/projected/2dc05f89-a229-4b2b-9d29-4d3e2521b691-kube-api-access-99j6x\") pod \"multus-additional-cni-plugins-tkbq8\" (UID: \"2dc05f89-a229-4b2b-9d29-4d3e2521b691\") " pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.336647 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.336623    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5x9b\" (UniqueName: \"kubernetes.io/projected/ecbaf110-7713-4aed-b3ad-78f77508ceb5-kube-api-access-k5x9b\") pod \"node-resolver-tlm66\" (UID: \"ecbaf110-7713-4aed-b3ad-78f77508ceb5\") " pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.337765 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.337739    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcxfp\" (UniqueName: \"kubernetes.io/projected/8e5e8cfb-8273-44f3-ac0c-fce49b77c268-kube-api-access-jcxfp\") pod \"ovnkube-node-xw7vr\" (UID: \"8e5e8cfb-8273-44f3-ac0c-fce49b77c268\") " pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.358162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.358137    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:53:32.415695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.415659    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vbghm"
Jul 29 17:53:32.423622 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.423587    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2"
Jul 29 17:53:32.435671 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:32.435641    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44570463_5430_45f8_9cdb_c8ea4fa2927a.slice/crio-3715b6e3d6491592886acd63b9ed7fc071c32d3875b6314d1d40c126c3e02545 WatchSource:0}: Error finding container 3715b6e3d6491592886acd63b9ed7fc071c32d3875b6314d1d40c126c3e02545: Status 404 returned error can't find the container with id 3715b6e3d6491592886acd63b9ed7fc071c32d3875b6314d1d40c126c3e02545
Jul 29 17:53:32.437914 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.437887    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:32.439367 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:32.438806    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3e7647_6d69_4cb0_b046_5fa4fccb1bfd.slice/crio-e48933b6fb0bc7a21209f000e7d98bb3951451b77eb1492e20c3afebebe6c8b1 WatchSource:0}: Error finding container e48933b6fb0bc7a21209f000e7d98bb3951451b77eb1492e20c3afebebe6c8b1: Status 404 returned error can't find the container with id e48933b6fb0bc7a21209f000e7d98bb3951451b77eb1492e20c3afebebe6c8b1
Jul 29 17:53:32.444126 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.444104    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tlm66"
Jul 29 17:53:32.448139 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:32.448105    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fbc1a2_3b7b_44ac_8cba_d6f2b6753d12.slice/crio-6932bb9f9ab877d1fbc7a4444d44e6832d2fb5fe19fd6e257b3ed2a0d4212f26 WatchSource:0}: Error finding container 6932bb9f9ab877d1fbc7a4444d44e6832d2fb5fe19fd6e257b3ed2a0d4212f26: Status 404 returned error can't find the container with id 6932bb9f9ab877d1fbc7a4444d44e6832d2fb5fe19fd6e257b3ed2a0d4212f26
Jul 29 17:53:32.451743 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.451718    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rkcjz"
Jul 29 17:53:32.461469 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.461438    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tkbq8"
Jul 29 17:53:32.469380 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.469358    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g9f6g"
Jul 29 17:53:32.477467 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.477442    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:32.486418 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.486398    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tsnm5"
Jul 29 17:53:32.724770 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.724691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:32.724770 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.724734    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:32.724971 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.724852    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:32.724971 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.724858    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:32.724971 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.724924    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret podName:df55af57-df49-40aa-9e88-a6f8e1a604cf nodeName:}" failed. No retries permitted until 2026-07-29 17:53:33.724904069 +0000 UTC m=+3.966865099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret") pod "global-pull-secret-syncer-bphzv" (UID: "df55af57-df49-40aa-9e88-a6f8e1a604cf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:32.724971 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.724944    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs podName:3724628d-c3ad-47a9-be69-337160e0fe78 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:33.724933886 +0000 UTC m=+3.966894918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs") pod "network-metrics-daemon-kqpsv" (UID: "3724628d-c3ad-47a9-be69-337160e0fe78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:32.825386 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:32.825341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:32.825560 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.825467    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:53:32.825560 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.825483    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:53:32.825560 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.825491    2560 projected.go:196] Error preparing data for projected volume kube-api-access-wckcp for pod openshift-network-diagnostics/network-check-target-k9wdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:32.825560 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:32.825542    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp podName:d494b6bf-3e7d-4273-90bc-80b156c5a16e nodeName:}" failed. No retries permitted until 2026-07-29 17:53:33.825527914 +0000 UTC m=+4.067488930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wckcp" (UniqueName: "kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp") pod "network-check-target-k9wdg" (UID: "d494b6bf-3e7d-4273-90bc-80b156c5a16e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:33.163457 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.163353    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:48:32 +0000 UTC" deadline="2028-03-30 11:12:33.207533097 +0000 UTC"
Jul 29 17:53:33.163457 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.163424    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14633h19m0.044117984s"
Jul 29 17:53:33.276121 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.276087    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:33.276292 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.276197    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:33.283722 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.283689    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ksc6p" event={"ID":"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12","Type":"ContainerStarted","Data":"6932bb9f9ab877d1fbc7a4444d44e6832d2fb5fe19fd6e257b3ed2a0d4212f26"}
Jul 29 17:53:33.284897 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.284873    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2" event={"ID":"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd","Type":"ContainerStarted","Data":"e48933b6fb0bc7a21209f000e7d98bb3951451b77eb1492e20c3afebebe6c8b1"}
Jul 29 17:53:33.286096 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.286073    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vbghm" event={"ID":"44570463-5430-45f8-9cdb-c8ea4fa2927a","Type":"ContainerStarted","Data":"3715b6e3d6491592886acd63b9ed7fc071c32d3875b6314d1d40c126c3e02545"}
Jul 29 17:53:33.457250 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:33.457224    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e5e8cfb_8273_44f3_ac0c_fce49b77c268.slice/crio-81c45e881a368b1c9de850fea253d6c1fb4beb9fb98f3d75981a59ef95fa938a WatchSource:0}: Error finding container 81c45e881a368b1c9de850fea253d6c1fb4beb9fb98f3d75981a59ef95fa938a: Status 404 returned error can't find the container with id 81c45e881a368b1c9de850fea253d6c1fb4beb9fb98f3d75981a59ef95fa938a
Jul 29 17:53:33.458410 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:33.458384    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc05f89_a229_4b2b_9d29_4d3e2521b691.slice/crio-10420320bc413480bb3056f31992a20c2e46a32a62a304798152cd0f417c9d72 WatchSource:0}: Error finding container 10420320bc413480bb3056f31992a20c2e46a32a62a304798152cd0f417c9d72: Status 404 returned error can't find the container with id 10420320bc413480bb3056f31992a20c2e46a32a62a304798152cd0f417c9d72
Jul 29 17:53:33.459714 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:33.459684    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a9391d_00df_4c40_80cc_137625065288.slice/crio-b5726e5dcae26fd943c677518cce272285e8b78e91db12c420e37a5d4d2eeb68 WatchSource:0}: Error finding container b5726e5dcae26fd943c677518cce272285e8b78e91db12c420e37a5d4d2eeb68: Status 404 returned error can't find the container with id b5726e5dcae26fd943c677518cce272285e8b78e91db12c420e37a5d4d2eeb68
Jul 29 17:53:33.461157 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:53:33.460946    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073dff5b_1210_4a03_8917_cad6c1e91f46.slice/crio-7bddc1e3139dac7111445bceb0b31f766165d82580838762511db21db03e4e87 WatchSource:0}: Error finding container 7bddc1e3139dac7111445bceb0b31f766165d82580838762511db21db03e4e87: Status 404 returned error can't find the container with id 7bddc1e3139dac7111445bceb0b31f766165d82580838762511db21db03e4e87
Jul 29 17:53:33.731955 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.731728    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:33.732063 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.732024    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:33.732140 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.732131    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:33.732197 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.732184    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret podName:df55af57-df49-40aa-9e88-a6f8e1a604cf nodeName:}" failed. No retries permitted until 2026-07-29 17:53:35.732168946 +0000 UTC m=+5.974129962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret") pod "global-pull-secret-syncer-bphzv" (UID: "df55af57-df49-40aa-9e88-a6f8e1a604cf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:33.732569 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.732542    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:33.732653 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.732601    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs podName:3724628d-c3ad-47a9-be69-337160e0fe78 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:35.732587816 +0000 UTC m=+5.974548835 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs") pod "network-metrics-daemon-kqpsv" (UID: "3724628d-c3ad-47a9-be69-337160e0fe78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:33.833842 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:33.833184    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:33.833842 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.833420    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:53:33.833842 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.833441    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:53:33.833842 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.833454    2560 projected.go:196] Error preparing data for projected volume kube-api-access-wckcp for pod openshift-network-diagnostics/network-check-target-k9wdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:33.833842 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:33.833512    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp podName:d494b6bf-3e7d-4273-90bc-80b156c5a16e nodeName:}" failed. No retries permitted until 2026-07-29 17:53:35.833494986 +0000 UTC m=+6.075456006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wckcp" (UniqueName: "kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp") pod "network-check-target-k9wdg" (UID: "d494b6bf-3e7d-4273-90bc-80b156c5a16e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:34.164496 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.164393    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:48:32 +0000 UTC" deadline="2027-12-23 01:08:48.259929178 +0000 UTC"
Jul 29 17:53:34.164496 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.164424    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12271h15m14.095509357s"
Jul 29 17:53:34.276411 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.276379    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:34.276568 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:34.276527    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:34.276979 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.276936    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:34.277140 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:34.277064    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:34.291975 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.291914    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tsnm5" event={"ID":"073dff5b-1210-4a03-8917-cad6c1e91f46","Type":"ContainerStarted","Data":"7bddc1e3139dac7111445bceb0b31f766165d82580838762511db21db03e4e87"}
Jul 29 17:53:34.293694 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.293666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g9f6g" event={"ID":"25a9391d-00df-4c40-80cc-137625065288","Type":"ContainerStarted","Data":"b5726e5dcae26fd943c677518cce272285e8b78e91db12c420e37a5d4d2eeb68"}
Jul 29 17:53:34.295759 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.295734    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rkcjz" event={"ID":"43f85418-dbf7-4a41-8ddf-cb461995157e","Type":"ContainerStarted","Data":"1f836d82654028fad009dce1441ee23037c05d9fd6fe87f081322c6dd4dfb0d0"}
Jul 29 17:53:34.299413 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.299388    2560 generic.go:356] "Generic (PLEG): container finished" podID="ff29b82ad41d8bf0cc74890e7834231a" containerID="6b68ed5fb2b65fecc7f8a46fd19e1d27636103e936bc162d77ee4701b38e5ae4" exitCode=0
Jul 29 17:53:34.299965 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.299942    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal" event={"ID":"ff29b82ad41d8bf0cc74890e7834231a","Type":"ContainerDied","Data":"6b68ed5fb2b65fecc7f8a46fd19e1d27636103e936bc162d77ee4701b38e5ae4"}
Jul 29 17:53:34.307244 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.307210    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerStarted","Data":"10420320bc413480bb3056f31992a20c2e46a32a62a304798152cd0f417c9d72"}
Jul 29 17:53:34.321256 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.321224    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"81c45e881a368b1c9de850fea253d6c1fb4beb9fb98f3d75981a59ef95fa938a"}
Jul 29 17:53:34.343730 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.340140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tlm66" event={"ID":"ecbaf110-7713-4aed-b3ad-78f77508ceb5","Type":"ContainerStarted","Data":"320d6ddeec7182325641f8aead644ea892d2617270b50a1554e895ff54295c95"}
Jul 29 17:53:34.363694 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:34.362982    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal" event={"ID":"a82b4de24054f43493689057761ee11c","Type":"ContainerStarted","Data":"a6a8c9b434211554337268e1b054ad08f96381c42dd413ca1162f7f9cf142fa3"}
Jul 29 17:53:35.276224 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:35.276187    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:35.276711 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.276335    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:35.377370 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:35.377334    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal" event={"ID":"ff29b82ad41d8bf0cc74890e7834231a","Type":"ContainerStarted","Data":"8a42a2e1e42790887f5af9fa4af75e6a516a290600946c46f5346025c6171c7c"}
Jul 29 17:53:35.392483 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:35.392422    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-240.ec2.internal" podStartSLOduration=4.392398433 podStartE2EDuration="4.392398433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:34.377170093 +0000 UTC m=+4.619131140" watchObservedRunningTime="2026-07-29 17:53:35.392398433 +0000 UTC m=+5.634359473"
Jul 29 17:53:35.754590 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:35.754223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:35.754590 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:35.754336    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:35.754590 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.754462    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:35.754590 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.754466    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:35.754590 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.754519    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret podName:df55af57-df49-40aa-9e88-a6f8e1a604cf nodeName:}" failed. No retries permitted until 2026-07-29 17:53:39.754501857 +0000 UTC m=+9.996462886 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret") pod "global-pull-secret-syncer-bphzv" (UID: "df55af57-df49-40aa-9e88-a6f8e1a604cf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:35.754590 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.754541    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs podName:3724628d-c3ad-47a9-be69-337160e0fe78 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:39.754528883 +0000 UTC m=+9.996489906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs") pod "network-metrics-daemon-kqpsv" (UID: "3724628d-c3ad-47a9-be69-337160e0fe78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:35.854836 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:35.854804    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:35.855005 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.854967    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:53:35.855005 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.854989    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:53:35.855005 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.855002    2560 projected.go:196] Error preparing data for projected volume kube-api-access-wckcp for pod openshift-network-diagnostics/network-check-target-k9wdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:35.855239 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:35.855058    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp podName:d494b6bf-3e7d-4273-90bc-80b156c5a16e nodeName:}" failed. No retries permitted until 2026-07-29 17:53:39.855038275 +0000 UTC m=+10.096999293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wckcp" (UniqueName: "kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp") pod "network-check-target-k9wdg" (UID: "d494b6bf-3e7d-4273-90bc-80b156c5a16e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:36.277732 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:36.277692    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:36.278220 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:36.277833    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:36.278220 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:36.277855    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:36.278220 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:36.277968    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:37.276630 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:37.276594    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:37.276793 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:37.276735    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:38.276048 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:38.276016    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:38.276443 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:38.276132    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:38.276505 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:38.276462    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:38.276587 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:38.276541    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:39.276850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:39.276203    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:39.276850 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.276362    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:39.789602 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:39.788860    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:39.789602 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:39.788935    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:39.789602 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.789041    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:39.789602 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.789103    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret podName:df55af57-df49-40aa-9e88-a6f8e1a604cf nodeName:}" failed. No retries permitted until 2026-07-29 17:53:47.789083232 +0000 UTC m=+18.031044268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret") pod "global-pull-secret-syncer-bphzv" (UID: "df55af57-df49-40aa-9e88-a6f8e1a604cf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:39.789602 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.789509    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:39.789602 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.789561    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs podName:3724628d-c3ad-47a9-be69-337160e0fe78 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:47.789545092 +0000 UTC m=+18.031506110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs") pod "network-metrics-daemon-kqpsv" (UID: "3724628d-c3ad-47a9-be69-337160e0fe78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:39.890201 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:39.890166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:39.890375 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.890317    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:53:39.890375 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.890340    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:53:39.890375 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.890352    2560 projected.go:196] Error preparing data for projected volume kube-api-access-wckcp for pod openshift-network-diagnostics/network-check-target-k9wdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:39.890520 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:39.890419    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp podName:d494b6bf-3e7d-4273-90bc-80b156c5a16e nodeName:}" failed. No retries permitted until 2026-07-29 17:53:47.890398537 +0000 UTC m=+18.132359571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wckcp" (UniqueName: "kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp") pod "network-check-target-k9wdg" (UID: "d494b6bf-3e7d-4273-90bc-80b156c5a16e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:40.277687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:40.277651    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:40.278137 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:40.277808    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:40.278137 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:40.278097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:40.278218 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:40.278169    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:41.276639 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:41.276132    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:41.276639 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:41.276258    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:42.278336 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:42.278307    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:42.278788 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:42.278438    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:42.278788 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:42.278500    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:42.278788 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:42.278569    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:43.275827 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:43.275784    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:43.276007 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:43.275920    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:44.275764 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:44.275730    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:44.276236 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:44.275864    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:44.276330 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:44.276301    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:44.276411 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:44.276392    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:45.276669 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:45.276630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:45.277080 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:45.276765    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:46.276022 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:46.275988    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:46.276164 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:46.276114    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:46.276214 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:46.276171    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:46.276308 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:46.276291    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:47.276419 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:47.276390    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:47.276751 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.276525    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:47.852413 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:47.852378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:47.852603 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:47.852428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:47.852603 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.852559    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:47.852603 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.852561    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:47.852715 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.852626    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs podName:3724628d-c3ad-47a9-be69-337160e0fe78 nodeName:}" failed. No retries permitted until 2026-07-29 17:54:03.852608008 +0000 UTC m=+34.094569027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs") pod "network-metrics-daemon-kqpsv" (UID: "3724628d-c3ad-47a9-be69-337160e0fe78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:47.852715 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.852642    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret podName:df55af57-df49-40aa-9e88-a6f8e1a604cf nodeName:}" failed. No retries permitted until 2026-07-29 17:54:03.852636085 +0000 UTC m=+34.094597101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret") pod "global-pull-secret-syncer-bphzv" (UID: "df55af57-df49-40aa-9e88-a6f8e1a604cf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:47.953589 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:47.953551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:47.953750 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.953729    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:53:47.953819 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.953754    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:53:47.953819 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.953764    2560 projected.go:196] Error preparing data for projected volume kube-api-access-wckcp for pod openshift-network-diagnostics/network-check-target-k9wdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:47.953908 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:47.953824    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp podName:d494b6bf-3e7d-4273-90bc-80b156c5a16e nodeName:}" failed. No retries permitted until 2026-07-29 17:54:03.953804877 +0000 UTC m=+34.195765896 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wckcp" (UniqueName: "kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp") pod "network-check-target-k9wdg" (UID: "d494b6bf-3e7d-4273-90bc-80b156c5a16e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:48.275873 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:48.275839    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:48.276028 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:48.275967    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:48.276069 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:48.276023    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:48.276164 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:48.276137    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:49.276169 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:49.276135    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:49.276649 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:49.276253    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:50.277105 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:50.277066    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:50.277557 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:50.277190    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:50.277557 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:50.277248    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:50.277557 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:50.277364    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:51.276494 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:51.276455    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:51.276686 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:51.276571    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:52.276218 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.275913    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:52.277015 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.275949    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:52.277015 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:52.276378    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:52.277015 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:52.276582    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:52.416015 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.415977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tlm66" event={"ID":"ecbaf110-7713-4aed-b3ad-78f77508ceb5","Type":"ContainerStarted","Data":"bf95eb6c928639fd2f8bfc6530cf5d06aef1efc702abac55342ca44c70cb7464"}
Jul 29 17:53:52.420110 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.420082    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ksc6p" event={"ID":"16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12","Type":"ContainerStarted","Data":"498133ee8685dc91594071423bdb840809a4c0f27e40f368e3739bf17e7ca067"}
Jul 29 17:53:52.421488 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.421466    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2" event={"ID":"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd","Type":"ContainerStarted","Data":"312448ac98bb1752c814b73066d654f96300337f2341aac61d1a29ecb4a7694f"}
Jul 29 17:53:52.422702 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.422678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g9f6g" event={"ID":"25a9391d-00df-4c40-80cc-137625065288","Type":"ContainerStarted","Data":"4f2c81cb03e5bec1d6dfed1ca1659c4cb251337a1e01f3aa6236d15a256a4fa3"}
Jul 29 17:53:52.424034 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.424003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rkcjz" event={"ID":"43f85418-dbf7-4a41-8ddf-cb461995157e","Type":"ContainerStarted","Data":"0c76a411e5215e5e763bb74f03835610c015377348dea2a8971f42e1edd12aa4"}
Jul 29 17:53:52.425302 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.425283    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vbghm" event={"ID":"44570463-5430-45f8-9cdb-c8ea4fa2927a","Type":"ContainerStarted","Data":"b9b9eae64051071f90033d7f0eea3f51df0314383f66c6d98ed4cb4f136f0641"}
Jul 29 17:53:52.426641 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.426622    2560 generic.go:356] "Generic (PLEG): container finished" podID="2dc05f89-a229-4b2b-9d29-4d3e2521b691" containerID="7aedea0e644c25bff2d3a5339b7d763a9fdc5e7b272d13679b2c1f4527ebbde2" exitCode=0
Jul 29 17:53:52.426719 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.426689    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerDied","Data":"7aedea0e644c25bff2d3a5339b7d763a9fdc5e7b272d13679b2c1f4527ebbde2"}
Jul 29 17:53:52.429225 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.429206    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 17:53:52.429654 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.429630    2560 generic.go:356] "Generic (PLEG): container finished" podID="8e5e8cfb-8273-44f3-ac0c-fce49b77c268" containerID="273d2cf68f08f5c666a123902498c144f3a45d1a0ee54ebdf7918f59faaced60" exitCode=1
Jul 29 17:53:52.429735 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.429666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"9b9b6f0b15583422ec08c0c165b79e9d123a31fecbd3e01fda81e780ec6af454"}
Jul 29 17:53:52.429735 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.429729    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"a60eea8f459980112dc7d1566ebaf47ee146f08addd6f6ce1550567cde04d7d2"}
Jul 29 17:53:52.429810 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.429777    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"48e5ec157766db3c49c79b7323d9a36a25b18f84cc2f0d25881ddc4ec1fac989"}
Jul 29 17:53:52.429850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.429824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerDied","Data":"273d2cf68f08f5c666a123902498c144f3a45d1a0ee54ebdf7918f59faaced60"}
Jul 29 17:53:52.429890 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.429872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"101a1db7d31cb45182c5f40a61bd18caa624dbf0304d444eee8af05ffce7569d"}
Jul 29 17:53:52.431090 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.431046    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-240.ec2.internal" podStartSLOduration=21.431010502 podStartE2EDuration="21.431010502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:35.392790065 +0000 UTC m=+5.634751106" watchObservedRunningTime="2026-07-29 17:53:52.431010502 +0000 UTC m=+22.672971537"
Jul 29 17:53:52.431686 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.431651    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tlm66" podStartSLOduration=4.118938703 podStartE2EDuration="22.431625704s" totalImagesPullingTime="18.312687001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:52.430972813 +0000 UTC m=+22.672933853" watchObservedRunningTime="2026-07-29 17:53:52.431625704 +0000 UTC m=+22.673586745"
Jul 29 17:53:52.446512 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.446468    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rkcjz" podStartSLOduration=4.135841883 podStartE2EDuration="22.446455239s" totalImagesPullingTime="18.310613356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:52.446021866 +0000 UTC m=+22.687982904" watchObservedRunningTime="2026-07-29 17:53:52.446455239 +0000 UTC m=+22.688416278"
Jul 29 17:53:52.461651 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.461613    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vbghm" podStartSLOduration=3.121269509 podStartE2EDuration="22.461599642s" totalImagesPullingTime="19.340330133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:52.46158476 +0000 UTC m=+22.703545798" watchObservedRunningTime="2026-07-29 17:53:52.461599642 +0000 UTC m=+22.703560674"
Jul 29 17:53:52.497472 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.497422    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g9f6g" podStartSLOduration=4.197831664 podStartE2EDuration="22.497408116s" totalImagesPullingTime="18.299576452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:52.497302802 +0000 UTC m=+22.739263834" watchObservedRunningTime="2026-07-29 17:53:52.497408116 +0000 UTC m=+22.739369155"
Jul 29 17:53:52.512164 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:52.512121    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ksc6p" podStartSLOduration=7.704498367 podStartE2EDuration="22.512107471s" totalImagesPullingTime="14.807609104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:52.511486048 +0000 UTC m=+22.753447105" watchObservedRunningTime="2026-07-29 17:53:52.512107471 +0000 UTC m=+22.754068556"
Jul 29 17:53:53.276125 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:53.276094    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:53.276330 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:53.276199    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:53.433878 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:53.433847    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tsnm5" event={"ID":"073dff5b-1210-4a03-8917-cad6c1e91f46","Type":"ContainerStarted","Data":"418774c2e37e20bf1a974e1e51d6ec76ee5098a358125e25528ababad28d13f8"}
Jul 29 17:53:53.437217 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:53.437184    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 17:53:53.439154 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:53.438807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"f5603c56f9efa5d202b689042f2ce37a375e84e58e2487aa1aaa3876cefc6a5a"}
Jul 29 17:53:53.447102 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:53.447058    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tsnm5" podStartSLOduration=5.177646595 podStartE2EDuration="23.447044503s" totalImagesPullingTime="18.269397908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:53.446984893 +0000 UTC m=+23.688945932" watchObservedRunningTime="2026-07-29 17:53:53.447044503 +0000 UTC m=+23.689005542"
Jul 29 17:53:53.479417 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:53.479391    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 17:53:54.180467 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:54.180345    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T17:53:53.479413321Z","UUID":"1023de7c-4104-4b68-8595-06fa93b09a38","Handler":null,"Name":"","Endpoint":""}
Jul 29 17:53:54.182106 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:54.182084    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 17:53:54.182457 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:54.182124    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 17:53:54.276398 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:54.276367    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:54.276927 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:54.276494    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:54.276971 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:54.276934    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:54.277065 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:54.277041    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:54.442173 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:54.442082    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2" event={"ID":"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd","Type":"ContainerStarted","Data":"e5116fa74efab57a09a4e034aa0705da842a1ad0d185435edf10e6a2281c0bf2"}
Jul 29 17:53:55.276053 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:55.276020    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:55.276239 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:55.276152    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:55.448396 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:55.448368    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 17:53:55.448831 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:55.448781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"1bea7d0b4260ce5e23bfd817f7e7604375e3aab145126cfb9b52f4cb294be781"}
Jul 29 17:53:55.451060 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:55.451029    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2" event={"ID":"1d3e7647-6d69-4cb0-b046-5fa4fccb1bfd","Type":"ContainerStarted","Data":"e7bd7d1951123136bcd626a779d9bc4fbd507bbaf1df00289f04925ff2a2360b"}
Jul 29 17:53:55.466786 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:55.466735    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqxl2" podStartSLOduration=8.263839696 podStartE2EDuration="25.466713264s" totalImagesPullingTime="17.202873568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:55.466060159 +0000 UTC m=+25.708021196" watchObservedRunningTime="2026-07-29 17:53:55.466713264 +0000 UTC m=+25.708674302"
Jul 29 17:53:56.276169 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:56.276135    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:56.276364 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:56.276139    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:56.276364 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:56.276297    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:56.276443 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:56.276365    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:56.283540 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:56.283504    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:56.284177 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:56.284148    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:56.453616 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:56.453577    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:56.454226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:56.454205    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ksc6p"
Jul 29 17:53:57.276853 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:57.276677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:57.277003 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:57.276929    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:57.456610 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:57.456533    2560 generic.go:356] "Generic (PLEG): container finished" podID="2dc05f89-a229-4b2b-9d29-4d3e2521b691" containerID="75216b55d7fcb81ddb2f8d51e5e21a4374c83cca6b800f76ddfc9fe0634270da" exitCode=0
Jul 29 17:53:57.457365 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:57.456622    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerDied","Data":"75216b55d7fcb81ddb2f8d51e5e21a4374c83cca6b800f76ddfc9fe0634270da"}
Jul 29 17:53:57.460284 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:57.460250    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 17:53:57.460705 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:57.460683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"55a348f4d108ffc4ce3d04f9e84f329fd3f8c56cedb51bfbf072c265bc13d66a"}
Jul 29 17:53:57.461100 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:57.461082    2560 scope.go:117] "RemoveContainer" containerID="273d2cf68f08f5c666a123902498c144f3a45d1a0ee54ebdf7918f59faaced60"
Jul 29 17:53:58.277289 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.276862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:58.277289 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:58.277000    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:58.280899 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.278395    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:58.280899 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:58.278566    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:58.466018 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.465992    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 17:53:58.466455 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.466422    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" event={"ID":"8e5e8cfb-8273-44f3-ac0c-fce49b77c268","Type":"ContainerStarted","Data":"3a2d9f00e3b92adae796680fedc45237dd91bcbfb3f60b6eb21cbcf53b53765c"}
Jul 29 17:53:58.466750 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.466717    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:58.466881 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.466791    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:58.466881 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.466840    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:58.482126 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.482101    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:58.482357 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.482340    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:53:58.519104 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.516972    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr" podStartSLOduration=10.152334652 podStartE2EDuration="28.516951807s" totalImagesPullingTime="18.364617155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:58.516856291 +0000 UTC m=+28.758817336" watchObservedRunningTime="2026-07-29 17:53:58.516951807 +0000 UTC m=+28.758912846"
Jul 29 17:53:58.768070 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.767996    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tlm66_ecbaf110-7713-4aed-b3ad-78f77508ceb5/dns-node-resolver/0.log"
Jul 29 17:53:58.980557 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.980516    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bphzv"]
Jul 29 17:53:58.980732 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.980693    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:53:58.980841 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:58.980817    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:53:58.985423 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.985394    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k9wdg"]
Jul 29 17:53:58.985525 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.985485    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:53:58.985571 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:58.985553    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:53:58.995529 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.995450    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kqpsv"]
Jul 29 17:53:58.995644 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:58.995560    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:53:58.995689 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:53:58.995656    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:53:59.470826 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:59.470727    2560 generic.go:356] "Generic (PLEG): container finished" podID="2dc05f89-a229-4b2b-9d29-4d3e2521b691" containerID="3641539dcc7942d242b9c555ca6c5ba6533a0c7a6e3333daac043801fe5e6fba" exitCode=0
Jul 29 17:53:59.471242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:59.470816    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerDied","Data":"3641539dcc7942d242b9c555ca6c5ba6533a0c7a6e3333daac043801fe5e6fba"}
Jul 29 17:53:59.544099 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:53:59.544074    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g9f6g_25a9391d-00df-4c40-80cc-137625065288/node-ca/0.log"
Jul 29 17:54:00.277578 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:00.277542    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:00.277727 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:00.277672    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:01.276629 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:01.276453    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:01.277020 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:01.276516    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:01.277020 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:01.276708    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:01.277020 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:01.276782    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:01.477859 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:01.477824    2560 generic.go:356] "Generic (PLEG): container finished" podID="2dc05f89-a229-4b2b-9d29-4d3e2521b691" containerID="166c41d31fe65f5731d7a0b389dcca110443470afb2f633c52ca11ed24bf70e9" exitCode=0
Jul 29 17:54:01.478009 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:01.477890    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerDied","Data":"166c41d31fe65f5731d7a0b389dcca110443470afb2f633c52ca11ed24bf70e9"}
Jul 29 17:54:02.275950 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:02.275910    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:02.276138 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:02.276031    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:03.276598 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:03.276562    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:03.277020 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:03.276607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:03.277020 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.276681    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:03.277020 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.276962    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:03.870570 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:03.870536    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:03.870846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:03.870580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:03.870846 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.870681    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:54:03.870846 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.870690    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:54:03.870846 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.870744    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs podName:3724628d-c3ad-47a9-be69-337160e0fe78 nodeName:}" failed. No retries permitted until 2026-07-29 17:54:35.87072975 +0000 UTC m=+66.112690766 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs") pod "network-metrics-daemon-kqpsv" (UID: "3724628d-c3ad-47a9-be69-337160e0fe78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:54:03.870846 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.870762    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret podName:df55af57-df49-40aa-9e88-a6f8e1a604cf nodeName:}" failed. No retries permitted until 2026-07-29 17:54:35.870752758 +0000 UTC m=+66.112713775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret") pod "global-pull-secret-syncer-bphzv" (UID: "df55af57-df49-40aa-9e88-a6f8e1a604cf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:54:03.971976 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:03.971936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:03.972235 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.972085    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:54:03.972235 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.972101    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:54:03.972235 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.972109    2560 projected.go:196] Error preparing data for projected volume kube-api-access-wckcp for pod openshift-network-diagnostics/network-check-target-k9wdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:54:03.972235 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:03.972169    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp podName:d494b6bf-3e7d-4273-90bc-80b156c5a16e nodeName:}" failed. No retries permitted until 2026-07-29 17:54:35.972150684 +0000 UTC m=+66.214111704 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wckcp" (UniqueName: "kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp") pod "network-check-target-k9wdg" (UID: "d494b6bf-3e7d-4273-90bc-80b156c5a16e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:54:04.275875 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:04.275842    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:04.276144 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:04.275988    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:05.276102 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:05.276070    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:05.276580 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:05.276074    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:05.276580 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:05.276187    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:05.276580 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:05.276315    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:06.275976 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:06.275947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:06.276129 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:06.276066    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:06.491514 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:06.491477    2560 generic.go:356] "Generic (PLEG): container finished" podID="2dc05f89-a229-4b2b-9d29-4d3e2521b691" containerID="06d0ec501270f5fd5181a0f334e602ce55409800a5739dc238b987a120fd8902" exitCode=0
Jul 29 17:54:06.491514 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:06.491515    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerDied","Data":"06d0ec501270f5fd5181a0f334e602ce55409800a5739dc238b987a120fd8902"}
Jul 29 17:54:07.276453 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:07.276418    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:07.276453 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:07.276451    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:07.276937 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:07.276566    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:07.276937 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:07.276680    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:07.497256 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:07.497225    2560 generic.go:356] "Generic (PLEG): container finished" podID="2dc05f89-a229-4b2b-9d29-4d3e2521b691" containerID="6237bd3ced073893309f181f6502e5ffe5f6e43af1dfd2d72da0d1abf69409af" exitCode=0
Jul 29 17:54:07.497452 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:07.497302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerDied","Data":"6237bd3ced073893309f181f6502e5ffe5f6e43af1dfd2d72da0d1abf69409af"}
Jul 29 17:54:08.276439 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:08.276406    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:08.276616 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:08.276535    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:08.502323 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:08.502284    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" event={"ID":"2dc05f89-a229-4b2b-9d29-4d3e2521b691","Type":"ContainerStarted","Data":"6029c7bd51fa32abc13477b453474c5e2660ae7a4b326b5b66686b70d4e9db73"}
Jul 29 17:54:08.523575 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:08.523532    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tkbq8" podStartSLOduration=8.226095137 podStartE2EDuration="38.523514936s" totalImagesPullingTime="30.297419799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:08.523082443 +0000 UTC m=+38.765043481" watchObservedRunningTime="2026-07-29 17:54:08.523514936 +0000 UTC m=+38.765476004"
Jul 29 17:54:09.276243 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:09.276206    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:09.276446 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:09.276206    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:09.276446 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:09.276367    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:09.276446 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:09.276383    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:10.276635 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:10.276604    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:10.277087 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:10.276704    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:11.275629 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:11.275596    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:11.275800 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:11.275596    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:11.275800 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:11.275692    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:11.275800 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:11.275787    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:12.276341 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:12.276308    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:12.276711 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:12.276419    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:13.275755 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:13.275723    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:13.275930 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:13.275723    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:13.275930 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:13.275823    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:13.275930 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:13.275897    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:14.276102 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:14.276074    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:14.276477 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:14.276181    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:15.276680 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:15.276642    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:15.277155 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:15.276642    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:15.277155 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:15.276792    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bphzv" podUID="df55af57-df49-40aa-9e88-a6f8e1a604cf"
Jul 29 17:54:15.277155 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:15.276815    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9wdg" podUID="d494b6bf-3e7d-4273-90bc-80b156c5a16e"
Jul 29 17:54:16.276459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.276375    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:16.276639 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:16.276588    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kqpsv" podUID="3724628d-c3ad-47a9-be69-337160e0fe78"
Jul 29 17:54:16.541885 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.541813    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-240.ec2.internal" event="NodeReady"
Jul 29 17:54:16.542212 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.541934    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 17:54:16.594420 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.594387    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cmfxl"]
Jul 29 17:54:16.610922 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.610889    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b5xhl"]
Jul 29 17:54:16.611080 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.611015    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.613284 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.613163    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 17:54:16.613284 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.613182    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qzmlr\""
Jul 29 17:54:16.613464 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.613442    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 17:54:16.613522 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.613478    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 17:54:16.613522 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.613485    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 17:54:16.626743 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.626722    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmfxl"]
Jul 29 17:54:16.626850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.626770    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b5xhl"]
Jul 29 17:54:16.626850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.626831    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.630076 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.630058    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 17:54:16.630201 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.630132    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 17:54:16.630358 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.630341    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pzsbz\""
Jul 29 17:54:16.691543 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.691513    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2847d"]
Jul 29 17:54:16.721946 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.721925    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2847d"]
Jul 29 17:54:16.722094 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.722004    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:16.723839 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.723811    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mh5gc\""
Jul 29 17:54:16.723931 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.723839    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 17:54:16.723931 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.723847    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 17:54:16.726986 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.724483    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 17:54:16.758544 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bcea560-650a-41e7-94cf-69189935446f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.758651 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758548    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/859ad4b1-4e5f-4677-990b-aa063d8d75ae-metrics-tls\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.758651 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859ad4b1-4e5f-4677-990b-aa063d8d75ae-tmp-dir\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.758764 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758664    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/859ad4b1-4e5f-4677-990b-aa063d8d75ae-config-volume\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.758764 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bcea560-650a-41e7-94cf-69189935446f-data-volume\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.758764 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59tv9\" (UniqueName: \"kubernetes.io/projected/859ad4b1-4e5f-4677-990b-aa063d8d75ae-kube-api-access-59tv9\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.758910 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bcea560-650a-41e7-94cf-69189935446f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.758910 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758838    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1bcea560-650a-41e7-94cf-69189935446f-crio-socket\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.758910 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.758879    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44jc2\" (UniqueName: \"kubernetes.io/projected/1bcea560-650a-41e7-94cf-69189935446f-kube-api-access-44jc2\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.859977 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.859892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/859ad4b1-4e5f-4677-990b-aa063d8d75ae-config-volume\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.859977 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.859948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15ec954f-65e8-4d47-b88a-4fd8d418ae26-cert\") pod \"ingress-canary-2847d\" (UID: \"15ec954f-65e8-4d47-b88a-4fd8d418ae26\") " pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:16.859977 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.859974    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bcea560-650a-41e7-94cf-69189935446f-data-volume\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.860226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.859992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59tv9\" (UniqueName: \"kubernetes.io/projected/859ad4b1-4e5f-4677-990b-aa063d8d75ae-kube-api-access-59tv9\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.860226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bcea560-650a-41e7-94cf-69189935446f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.860226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860070    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1bcea560-650a-41e7-94cf-69189935446f-crio-socket\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.860226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860122    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44jc2\" (UniqueName: \"kubernetes.io/projected/1bcea560-650a-41e7-94cf-69189935446f-kube-api-access-44jc2\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.860226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bcea560-650a-41e7-94cf-69189935446f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.860226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860184    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/859ad4b1-4e5f-4677-990b-aa063d8d75ae-metrics-tls\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.860226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860210    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hws7\" (UniqueName: \"kubernetes.io/projected/15ec954f-65e8-4d47-b88a-4fd8d418ae26-kube-api-access-9hws7\") pod \"ingress-canary-2847d\" (UID: \"15ec954f-65e8-4d47-b88a-4fd8d418ae26\") " pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:16.860596 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859ad4b1-4e5f-4677-990b-aa063d8d75ae-tmp-dir\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.860596 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1bcea560-650a-41e7-94cf-69189935446f-crio-socket\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.860596 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bcea560-650a-41e7-94cf-69189935446f-data-volume\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.860722 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/859ad4b1-4e5f-4677-990b-aa063d8d75ae-config-volume\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.860912 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.860889    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bcea560-650a-41e7-94cf-69189935446f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.864386 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.864365    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bcea560-650a-41e7-94cf-69189935446f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.867320 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.867293    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44jc2\" (UniqueName: \"kubernetes.io/projected/1bcea560-650a-41e7-94cf-69189935446f-kube-api-access-44jc2\") pod \"insights-runtime-extractor-cmfxl\" (UID: \"1bcea560-650a-41e7-94cf-69189935446f\") " pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.869622 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.869601    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859ad4b1-4e5f-4677-990b-aa063d8d75ae-tmp-dir\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.871691 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.871660    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59tv9\" (UniqueName: \"kubernetes.io/projected/859ad4b1-4e5f-4677-990b-aa063d8d75ae-kube-api-access-59tv9\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.871691 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.871661    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/859ad4b1-4e5f-4677-990b-aa063d8d75ae-metrics-tls\") pod \"dns-default-b5xhl\" (UID: \"859ad4b1-4e5f-4677-990b-aa063d8d75ae\") " pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.920231 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.920196    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmfxl"
Jul 29 17:54:16.935372 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.935347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:16.960619 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.960590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hws7\" (UniqueName: \"kubernetes.io/projected/15ec954f-65e8-4d47-b88a-4fd8d418ae26-kube-api-access-9hws7\") pod \"ingress-canary-2847d\" (UID: \"15ec954f-65e8-4d47-b88a-4fd8d418ae26\") " pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:16.960735 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.960687    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15ec954f-65e8-4d47-b88a-4fd8d418ae26-cert\") pod \"ingress-canary-2847d\" (UID: \"15ec954f-65e8-4d47-b88a-4fd8d418ae26\") " pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:16.963163 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.963142    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15ec954f-65e8-4d47-b88a-4fd8d418ae26-cert\") pod \"ingress-canary-2847d\" (UID: \"15ec954f-65e8-4d47-b88a-4fd8d418ae26\") " pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:16.968903 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:16.968883    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hws7\" (UniqueName: \"kubernetes.io/projected/15ec954f-65e8-4d47-b88a-4fd8d418ae26-kube-api-access-9hws7\") pod \"ingress-canary-2847d\" (UID: \"15ec954f-65e8-4d47-b88a-4fd8d418ae26\") " pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:17.032836 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.032802    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2847d"
Jul 29 17:54:17.099390 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.099160    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b5xhl"]
Jul 29 17:54:17.103065 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:17.103038    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859ad4b1_4e5f_4677_990b_aa063d8d75ae.slice/crio-f739bd406773501b518144a5b8cd56ae38773ae514720c7905690a811947242a WatchSource:0}: Error finding container f739bd406773501b518144a5b8cd56ae38773ae514720c7905690a811947242a: Status 404 returned error can't find the container with id f739bd406773501b518144a5b8cd56ae38773ae514720c7905690a811947242a
Jul 29 17:54:17.115632 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.115526    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmfxl"]
Jul 29 17:54:17.120261 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:17.120237    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bcea560_650a_41e7_94cf_69189935446f.slice/crio-3765af3ffd98bee4cd1a6cbe851692e057ae44abb07d2b73b1fe46fc1de36bd5 WatchSource:0}: Error finding container 3765af3ffd98bee4cd1a6cbe851692e057ae44abb07d2b73b1fe46fc1de36bd5: Status 404 returned error can't find the container with id 3765af3ffd98bee4cd1a6cbe851692e057ae44abb07d2b73b1fe46fc1de36bd5
Jul 29 17:54:17.184859 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.184826    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2847d"]
Jul 29 17:54:17.188419 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:17.188394    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ec954f_65e8_4d47_b88a_4fd8d418ae26.slice/crio-565b2b4c6d15aa58d9902a75890bd493ac8b34fbd23e26ae56119a8c86eb924a WatchSource:0}: Error finding container 565b2b4c6d15aa58d9902a75890bd493ac8b34fbd23e26ae56119a8c86eb924a: Status 404 returned error can't find the container with id 565b2b4c6d15aa58d9902a75890bd493ac8b34fbd23e26ae56119a8c86eb924a
Jul 29 17:54:17.276350 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.276322    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:17.276507 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.276363    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:17.278524 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.278497    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 17:54:17.278524 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.278513    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmgk6\""
Jul 29 17:54:17.278703 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.278552    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:54:17.278703 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.278497    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 17:54:17.523197 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.523151    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5xhl" event={"ID":"859ad4b1-4e5f-4677-990b-aa063d8d75ae","Type":"ContainerStarted","Data":"f739bd406773501b518144a5b8cd56ae38773ae514720c7905690a811947242a"}
Jul 29 17:54:17.524210 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.524184    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2847d" event={"ID":"15ec954f-65e8-4d47-b88a-4fd8d418ae26","Type":"ContainerStarted","Data":"565b2b4c6d15aa58d9902a75890bd493ac8b34fbd23e26ae56119a8c86eb924a"}
Jul 29 17:54:17.525330 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:17.525308    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmfxl" event={"ID":"1bcea560-650a-41e7-94cf-69189935446f","Type":"ContainerStarted","Data":"3765af3ffd98bee4cd1a6cbe851692e057ae44abb07d2b73b1fe46fc1de36bd5"}
Jul 29 17:54:18.276239 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:18.276205    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:18.279469 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:18.278622    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kkd97\""
Jul 29 17:54:18.279469 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:18.278853    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:54:19.906781 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.906745    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-cphv5"]
Jul 29 17:54:19.927331 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.927300    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-cphv5"]
Jul 29 17:54:19.927483 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.927387    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:19.929447 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.929423    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 17:54:19.929825 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.929806    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 17:54:19.930912 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.930003    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 17:54:19.930912 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.930217    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 17:54:19.930912 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.930413    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xlr2n\""
Jul 29 17:54:19.930912 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:19.930563    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 17:54:20.084800 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.084768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc509132-6ae5-4fdc-a3a1-a057c4a17832-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.084800 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.084805    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc95x\" (UniqueName: \"kubernetes.io/projected/fc509132-6ae5-4fdc-a3a1-a057c4a17832-kube-api-access-bc95x\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.085066 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.084897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc509132-6ae5-4fdc-a3a1-a057c4a17832-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.085066 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.084923    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc509132-6ae5-4fdc-a3a1-a057c4a17832-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.185373 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.185295    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc509132-6ae5-4fdc-a3a1-a057c4a17832-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.185373 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.185358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc509132-6ae5-4fdc-a3a1-a057c4a17832-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.185628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.185480    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc509132-6ae5-4fdc-a3a1-a057c4a17832-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.185628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.185528    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc95x\" (UniqueName: \"kubernetes.io/projected/fc509132-6ae5-4fdc-a3a1-a057c4a17832-kube-api-access-bc95x\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.186011 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.185987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc509132-6ae5-4fdc-a3a1-a057c4a17832-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.194769 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.194738    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc509132-6ae5-4fdc-a3a1-a057c4a17832-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.194882 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.194766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc509132-6ae5-4fdc-a3a1-a057c4a17832-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.194882 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.194841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc95x\" (UniqueName: \"kubernetes.io/projected/fc509132-6ae5-4fdc-a3a1-a057c4a17832-kube-api-access-bc95x\") pod \"prometheus-operator-84d69f794d-cphv5\" (UID: \"fc509132-6ae5-4fdc-a3a1-a057c4a17832\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:20.238565 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:20.238535    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5"
Jul 29 17:54:24.104966 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.104941    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-cphv5"]
Jul 29 17:54:24.108703 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:24.108677    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc509132_6ae5_4fdc_a3a1_a057c4a17832.slice/crio-827f7e048a1bb82c6c2347bb4e9e10217d29ad05128dee8513e6a23c05df761c WatchSource:0}: Error finding container 827f7e048a1bb82c6c2347bb4e9e10217d29ad05128dee8513e6a23c05df761c: Status 404 returned error can't find the container with id 827f7e048a1bb82c6c2347bb4e9e10217d29ad05128dee8513e6a23c05df761c
Jul 29 17:54:24.547042 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.546989    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2847d" event={"ID":"15ec954f-65e8-4d47-b88a-4fd8d418ae26","Type":"ContainerStarted","Data":"23f39648e0aa6d6fee0bb84016d17d65e801e5f4fd5019bcd27a5d0e58f7c537"}
Jul 29 17:54:24.548505 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.548476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmfxl" event={"ID":"1bcea560-650a-41e7-94cf-69189935446f","Type":"ContainerStarted","Data":"2e0467f6561cbf16c6042156e73a9b4199c492bf4bd5e026f34ba9be28efe79c"}
Jul 29 17:54:24.550039 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.550011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5xhl" event={"ID":"859ad4b1-4e5f-4677-990b-aa063d8d75ae","Type":"ContainerStarted","Data":"5fd23372de8752a90022cbce29737a60d3277e8aede7724cfc8988737ec198f4"}
Jul 29 17:54:24.550166 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.550066    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5xhl" event={"ID":"859ad4b1-4e5f-4677-990b-aa063d8d75ae","Type":"ContainerStarted","Data":"877fce2b25c56b46f36bfe581e25ecdaee803fe952cfb232a83d80d81d43dabd"}
Jul 29 17:54:24.550166 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.550098    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:24.551053 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.551030    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5" event={"ID":"fc509132-6ae5-4fdc-a3a1-a057c4a17832","Type":"ContainerStarted","Data":"827f7e048a1bb82c6c2347bb4e9e10217d29ad05128dee8513e6a23c05df761c"}
Jul 29 17:54:24.582622 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.582564    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2847d" podStartSLOduration=1.815472008 podStartE2EDuration="8.582543494s" totalImagesPullingTime="6.767071486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:24.565042368 +0000 UTC m=+54.807003406" watchObservedRunningTime="2026-07-29 17:54:24.582543494 +0000 UTC m=+54.824504532"
Jul 29 17:54:24.583292 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:24.583230    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b5xhl" podStartSLOduration=2.159334556 podStartE2EDuration="8.583218169s" totalImagesPullingTime="6.423883613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:24.582088551 +0000 UTC m=+54.824049589" watchObservedRunningTime="2026-07-29 17:54:24.583218169 +0000 UTC m=+54.825179201"
Jul 29 17:54:26.560680 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:26.560649    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5" event={"ID":"fc509132-6ae5-4fdc-a3a1-a057c4a17832","Type":"ContainerStarted","Data":"152721db27ec39009fb152d5c5a7dee23283f5b13dcf05feac1397cc3ab9aab9"}
Jul 29 17:54:26.560680 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:26.560699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5" event={"ID":"fc509132-6ae5-4fdc-a3a1-a057c4a17832","Type":"ContainerStarted","Data":"f801a3e619549724c9c5ac28d684e1ac7ba0f74f588923a559f0291c5269d65b"}
Jul 29 17:54:26.576428 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:26.576369    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-cphv5" podStartSLOduration=5.785709946 podStartE2EDuration="7.576351409s" totalImagesPullingTime="1.790641463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:26.575566114 +0000 UTC m=+56.817527153" watchObservedRunningTime="2026-07-29 17:54:26.576351409 +0000 UTC m=+56.818312450"
Jul 29 17:54:28.256347 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.256294    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"]
Jul 29 17:54:28.267931 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.267898    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sbcc9"]
Jul 29 17:54:28.268091 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.268004    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.270348 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.270314    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.271077 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.271057    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"]
Jul 29 17:54:28.271647 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.271626    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 17:54:28.271748 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.271635    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 17:54:28.271748 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.271683    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-g9xjt\""
Jul 29 17:54:28.274135 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.274110    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 17:54:28.274135 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.274123    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 17:54:28.274333 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.274127    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gr9c5\""
Jul 29 17:54:28.274693 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.274413    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 17:54:28.275800 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.275286    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"]
Jul 29 17:54:28.275800 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.275369    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.279657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.279516    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 17:54:28.279657 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.279607    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 17:54:28.279865 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.279695    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 17:54:28.279865 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.279804    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vzmlb\""
Jul 29 17:54:28.293665 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.293642    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"]
Jul 29 17:54:28.350766 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.350732    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27eb9b6d-9114-4046-bb88-205f77a163fe-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.350939 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.350773    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-sys\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.350939 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.350802    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t8sl\" (UniqueName: \"kubernetes.io/projected/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-kube-api-access-5t8sl\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.350939 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.350826    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-metrics-client-ca\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.350939 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.350906    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-root\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.351158 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.350964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.351158 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27eb9b6d-9114-4046-bb88-205f77a163fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.351158 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d803fc25-e93b-4d83-9257-05a40c2b3632-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.351158 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-textfile\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.351158 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9jx\" (UniqueName: \"kubernetes.io/projected/27eb9b6d-9114-4046-bb88-205f77a163fe-kube-api-access-vb9jx\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.351158 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351144    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-tls\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.351401 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351180    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d803fc25-e93b-4d83-9257-05a40c2b3632-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.351401 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351341    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-accelerators-collector-config\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.351401 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351380    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbvv5\" (UniqueName: \"kubernetes.io/projected/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-api-access-cbvv5\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.351616 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351413    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.351616 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351453    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-wtmp\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.351616 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351480    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.351616 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351510    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27eb9b6d-9114-4046-bb88-205f77a163fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.351616 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.351534    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.452687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452659    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-root\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.452806 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.452806 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452725    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27eb9b6d-9114-4046-bb88-205f77a163fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.452806 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d803fc25-e93b-4d83-9257-05a40c2b3632-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.452806 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-textfile\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.452806 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452788    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-root\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.452806 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9jx\" (UniqueName: \"kubernetes.io/projected/27eb9b6d-9114-4046-bb88-205f77a163fe-kube-api-access-vb9jx\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.453079 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452821    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-tls\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453079 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.452840    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d803fc25-e93b-4d83-9257-05a40c2b3632-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.453079 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:28.452945    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 17:54:28.453079 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:28.453017    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-tls podName:540c3ce9-8fb3-4ae7-8403-d3b704e63bb9 nodeName:}" failed. No retries permitted until 2026-07-29 17:54:28.952996415 +0000 UTC m=+59.194957431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-tls") pod "node-exporter-sbcc9" (UID: "540c3ce9-8fb3-4ae7-8403-d3b704e63bb9") : secret "node-exporter-tls" not found
Jul 29 17:54:28.453299 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-accelerators-collector-config\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453299 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-textfile\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453299 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453193    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d803fc25-e93b-4d83-9257-05a40c2b3632-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.453299 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453240    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbvv5\" (UniqueName: \"kubernetes.io/projected/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-api-access-cbvv5\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.453299 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.453539 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453324    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-wtmp\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453539 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.453539 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453504    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d803fc25-e93b-4d83-9257-05a40c2b3632-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.453539 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453466    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27eb9b6d-9114-4046-bb88-205f77a163fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.453726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.453726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27eb9b6d-9114-4046-bb88-205f77a163fe-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.453726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-wtmp\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-sys\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t8sl\" (UniqueName: \"kubernetes.io/projected/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-kube-api-access-5t8sl\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453684    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-sys\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.453726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-metrics-client-ca\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.454079 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.453729    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-accelerators-collector-config\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.454079 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:28.453823    2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 17:54:28.454079 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:28.453886    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-tls podName:d803fc25-e93b-4d83-9257-05a40c2b3632 nodeName:}" failed. No retries permitted until 2026-07-29 17:54:28.953869176 +0000 UTC m=+59.195830195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-zhg77" (UID: "d803fc25-e93b-4d83-9257-05a40c2b3632") : secret "kube-state-metrics-tls" not found
Jul 29 17:54:28.454212 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.454189    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.454355 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.454337    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-metrics-client-ca\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.454797 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.454775    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27eb9b6d-9114-4046-bb88-205f77a163fe-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.455641 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.455619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27eb9b6d-9114-4046-bb88-205f77a163fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.456139 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.456110    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.456227 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.456168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.456376 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.456360    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27eb9b6d-9114-4046-bb88-205f77a163fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.463504 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.463458    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9jx\" (UniqueName: \"kubernetes.io/projected/27eb9b6d-9114-4046-bb88-205f77a163fe-kube-api-access-vb9jx\") pod \"openshift-state-metrics-b8f49f5c6-t6ns4\" (UID: \"27eb9b6d-9114-4046-bb88-205f77a163fe\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.463762 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.463715    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbvv5\" (UniqueName: \"kubernetes.io/projected/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-api-access-cbvv5\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.464536 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.464514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t8sl\" (UniqueName: \"kubernetes.io/projected/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-kube-api-access-5t8sl\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.569485 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.569398    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmfxl" event={"ID":"1bcea560-650a-41e7-94cf-69189935446f","Type":"ContainerStarted","Data":"5a642817d7bd26298dbddcc99ef6e34de978c8e3f2851fcc9d70a5c52a257eb9"}
Jul 29 17:54:28.581662 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.581635    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"
Jul 29 17:54:28.708567 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.708526    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4"]
Jul 29 17:54:28.711386 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:28.711361    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27eb9b6d_9114_4046_bb88_205f77a163fe.slice/crio-79659d35ed425ae407511df94bc7c024b77b667f9b9c96331c1e910f6390da59 WatchSource:0}: Error finding container 79659d35ed425ae407511df94bc7c024b77b667f9b9c96331c1e910f6390da59: Status 404 returned error can't find the container with id 79659d35ed425ae407511df94bc7c024b77b667f9b9c96331c1e910f6390da59
Jul 29 17:54:28.957926 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.957894    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-tls\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.958122 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.957958    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:28.960423 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.960399    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/540c3ce9-8fb3-4ae7-8403-d3b704e63bb9-node-exporter-tls\") pod \"node-exporter-sbcc9\" (UID: \"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9\") " pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:28.960537 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:28.960475    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d803fc25-e93b-4d83-9257-05a40c2b3632-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-zhg77\" (UID: \"d803fc25-e93b-4d83-9257-05a40c2b3632\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:29.187779 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.187699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sbcc9"
Jul 29 17:54:29.192847 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.192824    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"
Jul 29 17:54:29.196800 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:29.196773    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540c3ce9_8fb3_4ae7_8403_d3b704e63bb9.slice/crio-daef50ab032fea7470cb84b6155b8a279d22feaff5c5e2c156dc3ac98ce6f794 WatchSource:0}: Error finding container daef50ab032fea7470cb84b6155b8a279d22feaff5c5e2c156dc3ac98ce6f794: Status 404 returned error can't find the container with id daef50ab032fea7470cb84b6155b8a279d22feaff5c5e2c156dc3ac98ce6f794
Jul 29 17:54:29.310451 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.310419    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:29.314067 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.314045    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.315998 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.315975    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 17:54:29.316383 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316357    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 17:54:29.316383 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316379    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 17:54:29.316614 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316366    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6fdxf\""
Jul 29 17:54:29.316614 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316559    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 17:54:29.316758 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316634    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 17:54:29.316758 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316729    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 17:54:29.316758 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316752    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 17:54:29.316864 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.316757    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 17:54:29.323919 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.323898    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 17:54:29.324189 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.324163    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77"]
Jul 29 17:54:29.327541 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.327521    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:29.361043 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361015    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-out\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361043 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361050    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361260 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361086    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361260 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361108    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361260 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361260 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361201    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361476 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361260    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361476 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz7q6\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-kube-api-access-hz7q6\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361476 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361476 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361476 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361442    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361619 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-volume\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.361619 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.361495    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-web-config\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463623 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463524    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463623 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz7q6\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-kube-api-access-hz7q6\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463659    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463683    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:29.463719    2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-volume\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463760    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-web-config\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.463837 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:54:29.463805    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls podName:9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7 nodeName:}" failed. No retries permitted until 2026-07-29 17:54:29.96378333 +0000 UTC m=+60.205744348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7") : secret "alertmanager-main-tls" not found
Jul 29 17:54:29.464407 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-out\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.464407 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463885    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.464407 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.464407 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463947    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.464407 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.463981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.464846 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.464753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.466013 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.465978    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.466372 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.466325    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.467226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.467203    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-web-config\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.467955 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.467912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.468495 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.468043    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.468495 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.468414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-out\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.469346 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.469211    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.469652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.469629    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.470578 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.470554    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.470667 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.470593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-volume\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.474647 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.474627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz7q6\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-kube-api-access-hz7q6\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.575841 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.575704    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4" event={"ID":"27eb9b6d-9114-4046-bb88-205f77a163fe","Type":"ContainerStarted","Data":"6db079e79271841b06b8661980e3a49f1f7e7956a3cb1fb2053ee8c950acc51e"}
Jul 29 17:54:29.575841 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.575776    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4" event={"ID":"27eb9b6d-9114-4046-bb88-205f77a163fe","Type":"ContainerStarted","Data":"4211042c39819b22524bc18028403ff6250d4d9a09fccf7de4fce9df6c1662cc"}
Jul 29 17:54:29.575841 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.575802    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4" event={"ID":"27eb9b6d-9114-4046-bb88-205f77a163fe","Type":"ContainerStarted","Data":"79659d35ed425ae407511df94bc7c024b77b667f9b9c96331c1e910f6390da59"}
Jul 29 17:54:29.577112 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.577080    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbcc9" event={"ID":"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9","Type":"ContainerStarted","Data":"daef50ab032fea7470cb84b6155b8a279d22feaff5c5e2c156dc3ac98ce6f794"}
Jul 29 17:54:29.578303 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.578255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77" event={"ID":"d803fc25-e93b-4d83-9257-05a40c2b3632","Type":"ContainerStarted","Data":"9a491380d046079bc7d1c0adebae89b2c9682a38a83378edb44d48e6ea14b5bd"}
Jul 29 17:54:29.967971 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.967895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:29.970978 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:29.970943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:30.221627 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.221541    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6bd4688476-s6rrt"]
Jul 29 17:54:30.224289 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.224252    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.230193 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.227411    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 17:54:30.230193 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.227647    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 17:54:30.231624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.230998    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 17:54:30.231624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.231248    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dua162eg46rgu\""
Jul 29 17:54:30.231624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.231445    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-wg6qk\""
Jul 29 17:54:30.231624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.231496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 17:54:30.231624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.231610    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 17:54:30.234154 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.233995    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6fdxf\""
Jul 29 17:54:30.240157 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.240084    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bd4688476-s6rrt"]
Jul 29 17:54:30.243556 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.243537    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:30.270004 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.269973    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.270162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.270016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-tls\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.270162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.270124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.270162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.270141    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-grpc-tls\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.270305 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.270169    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.270305 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.270193    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.270305 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.270212    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-metrics-client-ca\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.270305 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.270289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gfpc\" (UniqueName: \"kubernetes.io/projected/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-kube-api-access-4gfpc\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371465    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-grpc-tls\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371934 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371508    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371934 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371535    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371934 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371565    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-metrics-client-ca\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371934 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gfpc\" (UniqueName: \"kubernetes.io/projected/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-kube-api-access-4gfpc\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371934 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.371934 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.371687    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-tls\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.372678 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.372649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-metrics-client-ca\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.376427 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.376402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-tls\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.376945 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.376917    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.377052 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.376945    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.377052 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.377024    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.378714 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.378668    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.382810 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.382787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gfpc\" (UniqueName: \"kubernetes.io/projected/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-kube-api-access-4gfpc\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.383847 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.383805    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29557c5b-8b47-4cf6-b7e8-00a4e7e17cda-secret-grpc-tls\") pod \"thanos-querier-6bd4688476-s6rrt\" (UID: \"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda\") " pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:30.488282 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.488238    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xw7vr"
Jul 29 17:54:30.544645 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:30.544611    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:31.077102 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.076990    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:31.081658 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:31.081629    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2e4292_1dbd_4c13_b97d_e7c24e2f32c7.slice/crio-52c594a36d3e69a99d7e7005024ead75c55a5d8114801e48618e8af655850d10 WatchSource:0}: Error finding container 52c594a36d3e69a99d7e7005024ead75c55a5d8114801e48618e8af655850d10: Status 404 returned error can't find the container with id 52c594a36d3e69a99d7e7005024ead75c55a5d8114801e48618e8af655850d10
Jul 29 17:54:31.089257 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.089143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bd4688476-s6rrt"]
Jul 29 17:54:31.588457 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.588411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4" event={"ID":"27eb9b6d-9114-4046-bb88-205f77a163fe","Type":"ContainerStarted","Data":"c8f0b294553c54cf2998bc772a5f14f3ff453bd51da9f9a6abd47ee3c5485ec1"}
Jul 29 17:54:31.590718 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.590680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbcc9" event={"ID":"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9","Type":"ContainerStarted","Data":"de3d1387fbcc7eba9ead23778b9e158246331d0299002d9222e07bfddbe20b4a"}
Jul 29 17:54:31.592815 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.592784    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77" event={"ID":"d803fc25-e93b-4d83-9257-05a40c2b3632","Type":"ContainerStarted","Data":"5864c42edb94025f2efe2432fcdbf9f5a199e5da920e7137620f5e8a6dbb150b"}
Jul 29 17:54:31.592952 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.592836    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77" event={"ID":"d803fc25-e93b-4d83-9257-05a40c2b3632","Type":"ContainerStarted","Data":"9b022424f712eb3d15e652da0c9e6e830e3e460ea863d19c657951b4a03c6d02"}
Jul 29 17:54:31.593924 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.593900    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerStarted","Data":"52c594a36d3e69a99d7e7005024ead75c55a5d8114801e48618e8af655850d10"}
Jul 29 17:54:31.625459 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:31.625010    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-t6ns4" podStartSLOduration=1.56770579 podStartE2EDuration="3.624987705s" totalImagesPullingTime="2.057281915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:31.606514123 +0000 UTC m=+61.848475164" watchObservedRunningTime="2026-07-29 17:54:31.624987705 +0000 UTC m=+61.866948744"
Jul 29 17:54:32.666990 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.666936    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-75f468f5b5-xgb9v"]
Jul 29 17:54:32.686863 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.686833    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75f468f5b5-xgb9v"]
Jul 29 17:54:32.687024 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.686954    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.689375 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.689326    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 17:54:32.689991 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.689947    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cg29t5i1f124c\""
Jul 29 17:54:32.690397 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.690375    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 17:54:32.690523 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.690444    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 17:54:32.690587 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.690557    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-l4fpc\""
Jul 29 17:54:32.690645 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.690616    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 17:54:32.790452 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:32.790427    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29557c5b_8b47_4cf6_b7e8_00a4e7e17cda.slice/crio-20da8f5381bc66d6c58eef24ab57c922052166039efdbadee8d5935757dde2bd WatchSource:0}: Error finding container 20da8f5381bc66d6c58eef24ab57c922052166039efdbadee8d5935757dde2bd: Status 404 returned error can't find the container with id 20da8f5381bc66d6c58eef24ab57c922052166039efdbadee8d5935757dde2bd
Jul 29 17:54:32.795849 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.795827    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-secret-metrics-server-tls\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.795950 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.795859    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-metrics-server-audit-profiles\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.795950 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.795921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-audit-log\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.796061 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.795950    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxnm8\" (UniqueName: \"kubernetes.io/projected/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-kube-api-access-jxnm8\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.796061 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.795967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-secret-metrics-server-client-certs\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.796061 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.795982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.796061 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.795997    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-client-ca-bundle\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.896953 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.896920    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-audit-log\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897125 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.896966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxnm8\" (UniqueName: \"kubernetes.io/projected/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-kube-api-access-jxnm8\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897125 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.896996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-secret-metrics-server-client-certs\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897125 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.897021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897314 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.897152    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-client-ca-bundle\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897314 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.897211    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-secret-metrics-server-tls\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897314 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.897245    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-metrics-server-audit-profiles\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897465 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.897432    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-audit-log\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.897733 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.897714    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.898060 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.898040    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-metrics-server-audit-profiles\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.899790 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.899766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-client-ca-bundle\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.900145 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.900129    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-secret-metrics-server-tls\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.900220 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.900203    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-secret-metrics-server-client-certs\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.904290 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.904253    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxnm8\" (UniqueName: \"kubernetes.io/projected/c8a9380c-bdaa-4797-88d0-f8e5c542e94e-kube-api-access-jxnm8\") pod \"metrics-server-75f468f5b5-xgb9v\" (UID: \"c8a9380c-bdaa-4797-88d0-f8e5c542e94e\") " pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:32.999704 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:32.999661    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:33.012455 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.012422    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"]
Jul 29 17:54:33.015557 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.015534    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.017485 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.017463    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 17:54:33.017485 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.017482    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tks8b\""
Jul 29 17:54:33.027073 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.027047    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"]
Jul 29 17:54:33.100012 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.098647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95e69613-571d-4ed9-a3b9-6e3fbd8351be-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-nq72d\" (UID: \"95e69613-571d-4ed9-a3b9-6e3fbd8351be\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.100012 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.098695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfwtz\" (UniqueName: \"kubernetes.io/projected/95e69613-571d-4ed9-a3b9-6e3fbd8351be-kube-api-access-hfwtz\") pod \"monitoring-plugin-68b7fc9574-nq72d\" (UID: \"95e69613-571d-4ed9-a3b9-6e3fbd8351be\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.148578 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.148532    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75f468f5b5-xgb9v"]
Jul 29 17:54:33.152097 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:33.152069    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a9380c_bdaa_4797_88d0_f8e5c542e94e.slice/crio-596d2fb6bee0105f8134d6ec2e6a6d77e689c09afd328d4a4c8d20c6a180f0de WatchSource:0}: Error finding container 596d2fb6bee0105f8134d6ec2e6a6d77e689c09afd328d4a4c8d20c6a180f0de: Status 404 returned error can't find the container with id 596d2fb6bee0105f8134d6ec2e6a6d77e689c09afd328d4a4c8d20c6a180f0de
Jul 29 17:54:33.199371 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.199339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95e69613-571d-4ed9-a3b9-6e3fbd8351be-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-nq72d\" (UID: \"95e69613-571d-4ed9-a3b9-6e3fbd8351be\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.199525 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.199401    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfwtz\" (UniqueName: \"kubernetes.io/projected/95e69613-571d-4ed9-a3b9-6e3fbd8351be-kube-api-access-hfwtz\") pod \"monitoring-plugin-68b7fc9574-nq72d\" (UID: \"95e69613-571d-4ed9-a3b9-6e3fbd8351be\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.201914 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.201896    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95e69613-571d-4ed9-a3b9-6e3fbd8351be-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-nq72d\" (UID: \"95e69613-571d-4ed9-a3b9-6e3fbd8351be\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.206714 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.206690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfwtz\" (UniqueName: \"kubernetes.io/projected/95e69613-571d-4ed9-a3b9-6e3fbd8351be-kube-api-access-hfwtz\") pod \"monitoring-plugin-68b7fc9574-nq72d\" (UID: \"95e69613-571d-4ed9-a3b9-6e3fbd8351be\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.329177 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.329089    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:33.543563 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.543527    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"]
Jul 29 17:54:33.590417 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:33.590342    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e69613_571d_4ed9_a3b9_6e3fbd8351be.slice/crio-6612c4df2305125fa39f8de27ae64ead3081167662d2fe591b6b61d81402b519 WatchSource:0}: Error finding container 6612c4df2305125fa39f8de27ae64ead3081167662d2fe591b6b61d81402b519: Status 404 returned error can't find the container with id 6612c4df2305125fa39f8de27ae64ead3081167662d2fe591b6b61d81402b519
Jul 29 17:54:33.606012 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.605979    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d" event={"ID":"95e69613-571d-4ed9-a3b9-6e3fbd8351be","Type":"ContainerStarted","Data":"6612c4df2305125fa39f8de27ae64ead3081167662d2fe591b6b61d81402b519"}
Jul 29 17:54:33.609145 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.608564    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmfxl" event={"ID":"1bcea560-650a-41e7-94cf-69189935446f","Type":"ContainerStarted","Data":"6275e6eab4b23feceea7626b2a74a1834b80d2e2e2e15335672d2343ba996aa8"}
Jul 29 17:54:33.610570 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.610540    2560 generic.go:356] "Generic (PLEG): container finished" podID="540c3ce9-8fb3-4ae7-8403-d3b704e63bb9" containerID="de3d1387fbcc7eba9ead23778b9e158246331d0299002d9222e07bfddbe20b4a" exitCode=0
Jul 29 17:54:33.610664 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.610584    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbcc9" event={"ID":"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9","Type":"ContainerDied","Data":"de3d1387fbcc7eba9ead23778b9e158246331d0299002d9222e07bfddbe20b4a"}
Jul 29 17:54:33.612032 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.612014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v" event={"ID":"c8a9380c-bdaa-4797-88d0-f8e5c542e94e","Type":"ContainerStarted","Data":"596d2fb6bee0105f8134d6ec2e6a6d77e689c09afd328d4a4c8d20c6a180f0de"}
Jul 29 17:54:33.614549 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.614522    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77" event={"ID":"d803fc25-e93b-4d83-9257-05a40c2b3632","Type":"ContainerStarted","Data":"c44457e152ac6f66bd210449c0dfd6ce26868adf9b1b160bb6dad8952e3a9700"}
Jul 29 17:54:33.616036 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.616012    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerID="48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5" exitCode=0
Jul 29 17:54:33.616143 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.616098    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5"}
Jul 29 17:54:33.617362 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.617335    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" event={"ID":"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda","Type":"ContainerStarted","Data":"20da8f5381bc66d6c58eef24ab57c922052166039efdbadee8d5935757dde2bd"}
Jul 29 17:54:33.625802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.625752    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cmfxl" podStartSLOduration=2.098609236 podStartE2EDuration="17.625734467s" totalImagesPullingTime="15.527125231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:54:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:33.625142942 +0000 UTC m=+63.867104005" watchObservedRunningTime="2026-07-29 17:54:33.625734467 +0000 UTC m=+63.867695505"
Jul 29 17:54:33.643481 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:33.643420    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-zhg77" podStartSLOduration=4.086061752 podStartE2EDuration="5.643400379s" totalImagesPullingTime="1.557338627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:33.641843221 +0000 UTC m=+63.883804269" watchObservedRunningTime="2026-07-29 17:54:33.643400379 +0000 UTC m=+63.885361421"
Jul 29 17:54:34.495093 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.495049    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:54:34.499825 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.499034    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.504868 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.504843    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 17:54:34.505096 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505077    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 17:54:34.505327 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505294    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 17:54:34.505469 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505453    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 17:54:34.505612 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505589    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 17:54:34.505763 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505745    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 17:54:34.505831 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505778    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 17:54:34.505886 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505457    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 17:54:34.505886 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505863    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 17:54:34.505991 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.505885    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 17:54:34.506060 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.506020    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 17:54:34.506228 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.506209    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-skqvf\""
Jul 29 17:54:34.506350 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.506329    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 17:54:34.506410 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.506384    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1ponm0rtt4fr3\""
Jul 29 17:54:34.506500 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.506484    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 17:54:34.507793 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.507668    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 17:54:34.531430 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.531328    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:54:34.567527 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.567418    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b5xhl"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614392    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614459    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsf5m\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-kube-api-access-wsf5m\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614572    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614609    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614727    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.614768 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614752    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614803    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614893    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614939    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.614970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.615012    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.615071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.615111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.615153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.615200    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.615340 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.615235    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.629083 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.627320    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbcc9" event={"ID":"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9","Type":"ContainerStarted","Data":"b9a91754be406b6fee95fd50232f3b8fc78ca7ea7e3dd3216566ec9073a075f8"}
Jul 29 17:54:34.629083 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.627396    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbcc9" event={"ID":"540c3ce9-8fb3-4ae7-8403-d3b704e63bb9","Type":"ContainerStarted","Data":"e9c45860150468cb7f7a42427fff825c10ff5eceab57cfb6846b668d25515782"}
Jul 29 17:54:34.656625 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.653470    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sbcc9" podStartSLOduration=4.839171113 podStartE2EDuration="6.653446582s" totalImagesPullingTime="1.691156148s" totalInitContainerRuntime="123.119321ms" isStatefulPod=true podCreationTimestamp="2026-07-29 17:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:34.652711025 +0000 UTC m=+64.894672065" watchObservedRunningTime="2026-07-29 17:54:34.653446582 +0000 UTC m=+64.895407621"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.715906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.715958    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.715994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716208    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsf5m\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-kube-api-access-wsf5m\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716286    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716311    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716387    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716473    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716498    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716523    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716565    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716646    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.717195 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716700    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.718242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716728    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.718242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716786    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.718242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.716855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.718242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.717925    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.719842 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.719496    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.721862 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.720484    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.721862 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.720901    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.721862 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.721542    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.721862 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.721730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.724755 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.724074    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.726818 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.725942    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.726818 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.726771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.732081 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.732040    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.733501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.732677    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.733501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.732685    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.733501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.732809    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.733501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.733075    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.733501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.733163    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.733501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.733459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.734829 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.734375    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.735180 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.735133    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsf5m\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-kube-api-access-wsf5m\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.736226 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.736185    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.737822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.737795    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:34.819105 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:34.818593    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:35.936343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:35.936295    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:35.936932 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:35.936412    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:35.939247 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:35.939226    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:54:35.939453 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:35.939436    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:54:35.949391 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:35.949365    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df55af57-df49-40aa-9e88-a6f8e1a604cf-original-pull-secret\") pod \"global-pull-secret-syncer-bphzv\" (UID: \"df55af57-df49-40aa-9e88-a6f8e1a604cf\") " pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:35.950242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:35.950215    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3724628d-c3ad-47a9-be69-337160e0fe78-metrics-certs\") pod \"network-metrics-daemon-kqpsv\" (UID: \"3724628d-c3ad-47a9-be69-337160e0fe78\") " pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:35.992699 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:35.992665    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kkd97\""
Jul 29 17:54:36.001246 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.001219    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kqpsv"
Jul 29 17:54:36.037715 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.037678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:36.040228 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.040208    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 17:54:36.049961 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.049940    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 17:54:36.061455 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.061427    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wckcp\" (UniqueName: \"kubernetes.io/projected/d494b6bf-3e7d-4273-90bc-80b156c5a16e-kube-api-access-wckcp\") pod \"network-check-target-k9wdg\" (UID: \"d494b6bf-3e7d-4273-90bc-80b156c5a16e\") " pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:36.187678 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.187576    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmgk6\""
Jul 29 17:54:36.190008 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.189984    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bphzv"
Jul 29 17:54:36.196818 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.196789    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:36.644300 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.643852    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v" event={"ID":"c8a9380c-bdaa-4797-88d0-f8e5c542e94e","Type":"ContainerStarted","Data":"72d2becec9ffb96ef3d5345adb2e5618864c13c39cf320b145f9afc51ea7f71a"}
Jul 29 17:54:36.648638 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.647638    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerStarted","Data":"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"}
Jul 29 17:54:36.652366 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.652340    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" event={"ID":"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda","Type":"ContainerStarted","Data":"03f93f871d73447b944fe883c5d95f035dc3e20eec12336109dc8adb36a1dcb2"}
Jul 29 17:54:36.656092 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.656034    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d" event={"ID":"95e69613-571d-4ed9-a3b9-6e3fbd8351be","Type":"ContainerStarted","Data":"29f88fd51976abdbf78c1d6216dfae8ed6737cb2e83e11859fad282191c82c63"}
Jul 29 17:54:36.657837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.657778    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:36.659665 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.659613    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kqpsv"]
Jul 29 17:54:36.665684 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.665349    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d"
Jul 29 17:54:36.666825 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.666782    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v" podStartSLOduration=1.401922241 podStartE2EDuration="4.666765328s" totalImagesPullingTime="3.264843087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:36.665948305 +0000 UTC m=+66.907909344" watchObservedRunningTime="2026-07-29 17:54:36.666765328 +0000 UTC m=+66.908726360"
Jul 29 17:54:36.694917 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.693373    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-nq72d" podStartSLOduration=1.861889842 podStartE2EDuration="4.693352024s" totalImagesPullingTime="2.831462182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:36.69008653 +0000 UTC m=+66.932047569" watchObservedRunningTime="2026-07-29 17:54:36.693352024 +0000 UTC m=+66.935313063"
Jul 29 17:54:36.868889 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.868783    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bphzv"]
Jul 29 17:54:36.871408 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:36.871348    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf55af57_df49_40aa_9e88_a6f8e1a604cf.slice/crio-39c4113a8c4ea0b8803dcb7db8ddc85578c590ee5e83e6f82b4d16114c9f3c5f WatchSource:0}: Error finding container 39c4113a8c4ea0b8803dcb7db8ddc85578c590ee5e83e6f82b4d16114c9f3c5f: Status 404 returned error can't find the container with id 39c4113a8c4ea0b8803dcb7db8ddc85578c590ee5e83e6f82b4d16114c9f3c5f
Jul 29 17:54:36.871837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.871810    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k9wdg"]
Jul 29 17:54:36.875257 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:36.875106    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd494b6bf_3e7d_4273_90bc_80b156c5a16e.slice/crio-9aba42aae445c1da03a9a1836b8b0bfc72e83a226b8c67fdcbc4b3415dbf099d WatchSource:0}: Error finding container 9aba42aae445c1da03a9a1836b8b0bfc72e83a226b8c67fdcbc4b3415dbf099d: Status 404 returned error can't find the container with id 9aba42aae445c1da03a9a1836b8b0bfc72e83a226b8c67fdcbc4b3415dbf099d
Jul 29 17:54:36.879192 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:36.879166    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:54:36.883348 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:54:36.883223    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a73d81b_162e_4a3a_ab35_9d831e5af1aa.slice/crio-6c863352674900329c651934912cf186ec02bf0e19c9f68a6f822931fb10b559 WatchSource:0}: Error finding container 6c863352674900329c651934912cf186ec02bf0e19c9f68a6f822931fb10b559: Status 404 returned error can't find the container with id 6c863352674900329c651934912cf186ec02bf0e19c9f68a6f822931fb10b559
Jul 29 17:54:37.662721 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.662688    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kqpsv" event={"ID":"3724628d-c3ad-47a9-be69-337160e0fe78","Type":"ContainerStarted","Data":"448d1e52f808a588dcb76a62e4999050515f795249192bbc96036388defbb4c7"}
Jul 29 17:54:37.665400 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.665369    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerID="9fa98e1a95db6e27f1514853f6b6327b9d7a01ec28426b78d2fec6889e59e425" exitCode=0
Jul 29 17:54:37.665549 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.665458    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"9fa98e1a95db6e27f1514853f6b6327b9d7a01ec28426b78d2fec6889e59e425"}
Jul 29 17:54:37.665549 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.665534    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerStarted","Data":"6c863352674900329c651934912cf186ec02bf0e19c9f68a6f822931fb10b559"}
Jul 29 17:54:37.667866 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.667841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bphzv" event={"ID":"df55af57-df49-40aa-9e88-a6f8e1a604cf","Type":"ContainerStarted","Data":"39c4113a8c4ea0b8803dcb7db8ddc85578c590ee5e83e6f82b4d16114c9f3c5f"}
Jul 29 17:54:37.672010 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.671987    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerStarted","Data":"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"}
Jul 29 17:54:37.672118 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.672055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerStarted","Data":"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"}
Jul 29 17:54:37.672118 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.672095    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerStarted","Data":"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"}
Jul 29 17:54:37.672232 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.672134    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerStarted","Data":"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"}
Jul 29 17:54:37.674647 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.674621    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" event={"ID":"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda","Type":"ContainerStarted","Data":"ea751590a2b6ec77d94ca174e143dc874d22d62cff63cfd9b8c7d39a89ea90bb"}
Jul 29 17:54:37.674723 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.674693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" event={"ID":"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda","Type":"ContainerStarted","Data":"3eda1ae811550432f424bd3ffc7a4f7cc7b3f312fb4ca77d863caebd8d0e3564"}
Jul 29 17:54:37.677650 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:37.677629    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k9wdg" event={"ID":"d494b6bf-3e7d-4273-90bc-80b156c5a16e","Type":"ContainerStarted","Data":"9aba42aae445c1da03a9a1836b8b0bfc72e83a226b8c67fdcbc4b3415dbf099d"}
Jul 29 17:54:38.687979 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.687938    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerStarted","Data":"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"}
Jul 29 17:54:38.692398 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.692366    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" event={"ID":"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda","Type":"ContainerStarted","Data":"ffa22d90a49f94cd337a4b6f01a4b01151c2c0d0fcc16dce0d975578d3a15dc8"}
Jul 29 17:54:38.692529 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.692430    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" event={"ID":"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda","Type":"ContainerStarted","Data":"97431b0ca8ad2a64c1cdd91fc412c242e88e54b6a3e76dbd10fe0a85f24262e5"}
Jul 29 17:54:38.692529 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.692458    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" event={"ID":"29557c5b-8b47-4cf6-b7e8-00a4e7e17cda","Type":"ContainerStarted","Data":"65789dd973fc5374d02a96a34839f5da001624129a4a076a7c46f507f6bafb05"}
Jul 29 17:54:38.692529 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.692502    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:38.695407 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.695381    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kqpsv" event={"ID":"3724628d-c3ad-47a9-be69-337160e0fe78","Type":"ContainerStarted","Data":"e853a04e27f68a686d91a1586b1537a14d061d7a0c8b030112b830b2f08d4545"}
Jul 29 17:54:38.695537 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.695426    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kqpsv" event={"ID":"3724628d-c3ad-47a9-be69-337160e0fe78","Type":"ContainerStarted","Data":"132a13b8ab1bfd223355095e8551cd954aaac38ebc76b8a00284c87ae1da7e6f"}
Jul 29 17:54:38.714604 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.714538    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.478183526 podStartE2EDuration="9.714516647s" totalImagesPullingTime="6.219322401s" totalInitContainerRuntime="17.01072ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:38.713290167 +0000 UTC m=+68.955251206" watchObservedRunningTime="2026-07-29 17:54:38.714516647 +0000 UTC m=+68.956477687"
Jul 29 17:54:38.739461 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.739391    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt" podStartSLOduration=3.95294719 podStartE2EDuration="8.739371415s" totalImagesPullingTime="4.786424225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:38.738579367 +0000 UTC m=+68.980540405" watchObservedRunningTime="2026-07-29 17:54:38.739371415 +0000 UTC m=+68.981332455"
Jul 29 17:54:38.756442 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:38.756213    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kqpsv" podStartSLOduration=67.499546235 podStartE2EDuration="1m8.756196628s" totalImagesPullingTime="1.256650393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:38.753889425 +0000 UTC m=+68.995850466" watchObservedRunningTime="2026-07-29 17:54:38.756196628 +0000 UTC m=+68.998157667"
Jul 29 17:54:43.721669 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.721620    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bphzv" event={"ID":"df55af57-df49-40aa-9e88-a6f8e1a604cf","Type":"ContainerStarted","Data":"b0ed390e940e9a85298d95d898901c471ed213ed278744f92c2ef8a32f0fcca7"}
Jul 29 17:54:43.723630 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.723598    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k9wdg" event={"ID":"d494b6bf-3e7d-4273-90bc-80b156c5a16e","Type":"ContainerStarted","Data":"f2dc3352dc79718904b75a0da556c87dd5fa04899ead4c95efce64ec6dfb8987"}
Jul 29 17:54:43.723747 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.723659    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:54:43.726720 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.726686    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerStarted","Data":"09f33447760de40c2f7176fc18975b66667d61f215be70546cdac899675d8b00"}
Jul 29 17:54:43.726812 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.726762    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerStarted","Data":"47715d55f950dcbbfeaba8d757b478f3c4fe4fd090294b352cb8051820a0789f"}
Jul 29 17:54:43.726812 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.726804    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerStarted","Data":"dea456dea3bd51065e30a80533aef102cb2b421b31d3c5decc78c657cf72153e"}
Jul 29 17:54:43.726885 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.726844    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerStarted","Data":"95c470dcb39713c90ea4e4b8667c9575fe59eccaffb86c256b27b6ff3ba9ce41"}
Jul 29 17:54:43.742669 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.742609    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bphzv" podStartSLOduration=67.280253355 podStartE2EDuration="1m13.742574591s" totalImagesPullingTime="6.462321236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:43.742172618 +0000 UTC m=+73.984133658" watchObservedRunningTime="2026-07-29 17:54:43.742574591 +0000 UTC m=+73.984535630"
Jul 29 17:54:43.765223 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:43.765177    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k9wdg" podStartSLOduration=67.319325104 podStartE2EDuration="1m13.765159104s" totalImagesPullingTime="6.445834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:43.763715951 +0000 UTC m=+74.005676990" watchObservedRunningTime="2026-07-29 17:54:43.765159104 +0000 UTC m=+74.007120142"
Jul 29 17:54:44.710328 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:44.710296    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6bd4688476-s6rrt"
Jul 29 17:54:44.737065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:44.737011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerStarted","Data":"2ec4956ade96d5d8ab235e7ecbd2260277e0d65761a8bdcb1db61bc1977ef629"}
Jul 29 17:54:44.737547 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:44.737177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerStarted","Data":"110ea0d9eec97d026439bfc18d1dfc83574c53ffe20d19b11e6360d29507b54b"}
Jul 29 17:54:44.779761 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:44.779706    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.098319615 podStartE2EDuration="10.779688335s" totalImagesPullingTime="5.66416322s" totalInitContainerRuntime="17.2055ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:44.777626999 +0000 UTC m=+75.019588070" watchObservedRunningTime="2026-07-29 17:54:44.779688335 +0000 UTC m=+75.021649372"
Jul 29 17:54:44.819401 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:44.819367    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:53.000490 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:53.000351    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:54:53.001143 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:54:53.001127    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:55:13.005644 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:13.005616    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:55:13.009372 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:13.009350    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-75f468f5b5-xgb9v"
Jul 29 17:55:14.740005 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:14.739978    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k9wdg"
Jul 29 17:55:34.818817 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:34.818778    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:34.839386 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:34.839360    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:34.950688 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:34.950636    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:48.520333 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.520301    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:55:48.520855 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.520579    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="alertmanager" containerID="cri-o://93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86" gracePeriod=120
Jul 29 17:55:48.520855 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.520650    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-metric" containerID="cri-o://55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0" gracePeriod=120
Jul 29 17:55:48.520855 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.520721    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="prom-label-proxy" containerID="cri-o://0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e" gracePeriod=120
Jul 29 17:55:48.520855 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.520740    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="config-reloader" containerID="cri-o://3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976" gracePeriod=120
Jul 29 17:55:48.520855 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.520736    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-web" containerID="cri-o://2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18" gracePeriod=120
Jul 29 17:55:48.521180 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.520744    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy" containerID="cri-o://2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe" gracePeriod=120
Jul 29 17:55:48.991229 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991196    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerID="0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e" exitCode=0
Jul 29 17:55:48.991229 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991223    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerID="2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe" exitCode=0
Jul 29 17:55:48.991229 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991229    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerID="3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976" exitCode=0
Jul 29 17:55:48.991229 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991235    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerID="93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86" exitCode=0
Jul 29 17:55:48.991563 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"}
Jul 29 17:55:48.991563 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991376    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"}
Jul 29 17:55:48.991563 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991420    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"}
Jul 29 17:55:48.991563 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:48.991459    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"}
Jul 29 17:55:49.759792 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.759769    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:49.898302 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898175    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898302 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898217    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-tls-assets\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898302 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898244    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-main-db\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898302 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898291    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-volume\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898418    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-metrics-client-ca\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898458    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-out\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898493    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898518    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-trusted-ca-bundle\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898568    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz7q6\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-kube-api-access-hz7q6\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898863 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898638    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:55:49.898863 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898655    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-web-config\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898863 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898692    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-web\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898863 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898720    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.898863 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898739    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:49.898863 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.898762    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-cluster-tls-config\") pod \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\" (UID: \"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7\") "
Jul 29 17:55:49.899343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.899012    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:49.899343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.899036    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-main-db\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.899343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.899059    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-metrics-client-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.902852 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.902800    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:49.903291 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.903225    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:55:49.903420 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.903300    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-out" (OuterVolumeSpecName: "config-out") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:55:49.903420 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.903369    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:49.903529 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.903420    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-kube-api-access-hz7q6" (OuterVolumeSpecName: "kube-api-access-hz7q6") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "kube-api-access-hz7q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:55:49.903529 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.903464    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-volume" (OuterVolumeSpecName: "config-volume") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:49.903816 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.903524    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:49.903881 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.903825    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:49.907518 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.907488    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:49.913969 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.913939    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-web-config" (OuterVolumeSpecName: "web-config") pod "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" (UID: "9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:49.998287 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.998237    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerID="55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0" exitCode=0
Jul 29 17:55:49.998287 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.998282    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerID="2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18" exitCode=0
Jul 29 17:55:49.998534 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.998300    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"}
Jul 29 17:55:49.998534 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.998356    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:49.998534 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.998395    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"}
Jul 29 17:55:49.998534 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.998424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7","Type":"ContainerDied","Data":"52c594a36d3e69a99d7e7005024ead75c55a5d8114801e48618e8af655850d10"}
Jul 29 17:55:49.998534 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.998454    2560 scope.go:117] "RemoveContainer" containerID="0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999426    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz7q6\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-kube-api-access-hz7q6\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999460    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-web-config\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999471    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999481    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999529    2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-cluster-tls-config\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999540    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999553    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-tls-assets\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999564    2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-volume\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999571    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-config-out\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:49.999574 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999578    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-secret-alertmanager-main-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:50.000072 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:49.999587    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:50.008358 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.008189    2560 scope.go:117] "RemoveContainer" containerID="55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"
Jul 29 17:55:50.016642 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.016623    2560 scope.go:117] "RemoveContainer" containerID="2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"
Jul 29 17:55:50.023493 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.023456    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:55:50.025327 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.025305    2560 scope.go:117] "RemoveContainer" containerID="2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"
Jul 29 17:55:50.030909 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.030888    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:55:50.033211 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.033197    2560 scope.go:117] "RemoveContainer" containerID="3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"
Jul 29 17:55:50.040476 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.040456    2560 scope.go:117] "RemoveContainer" containerID="93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"
Jul 29 17:55:50.048142 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.048126    2560 scope.go:117] "RemoveContainer" containerID="48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5"
Jul 29 17:55:50.056069 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.056032    2560 scope.go:117] "RemoveContainer" containerID="0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"
Jul 29 17:55:50.056415 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:55:50.056388    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e\": container with ID starting with 0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e not found: ID does not exist" containerID="0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"
Jul 29 17:55:50.056522 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.056425    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"} err="failed to get container status \"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e\": rpc error: code = NotFound desc = could not find container \"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e\": container with ID starting with 0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e not found: ID does not exist"
Jul 29 17:55:50.056522 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.056473    2560 scope.go:117] "RemoveContainer" containerID="55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"
Jul 29 17:55:50.056758 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:55:50.056738    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0\": container with ID starting with 55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0 not found: ID does not exist" containerID="55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"
Jul 29 17:55:50.056825 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.056771    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"} err="failed to get container status \"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0\": rpc error: code = NotFound desc = could not find container \"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0\": container with ID starting with 55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0 not found: ID does not exist"
Jul 29 17:55:50.056825 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.056798    2560 scope.go:117] "RemoveContainer" containerID="2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"
Jul 29 17:55:50.057082 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:55:50.057057    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe\": container with ID starting with 2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe not found: ID does not exist" containerID="2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"
Jul 29 17:55:50.057148 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057091    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"} err="failed to get container status \"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe\": rpc error: code = NotFound desc = could not find container \"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe\": container with ID starting with 2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe not found: ID does not exist"
Jul 29 17:55:50.057148 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057110    2560 scope.go:117] "RemoveContainer" containerID="2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"
Jul 29 17:55:50.057230 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057218    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:55:50.057388 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:55:50.057363    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18\": container with ID starting with 2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18 not found: ID does not exist" containerID="2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"
Jul 29 17:55:50.057433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057394    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"} err="failed to get container status \"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18\": rpc error: code = NotFound desc = could not find container \"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18\": container with ID starting with 2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18 not found: ID does not exist"
Jul 29 17:55:50.057433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057409    2560 scope.go:117] "RemoveContainer" containerID="3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"
Jul 29 17:55:50.057562 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057546    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="init-config-reloader"
Jul 29 17:55:50.057620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057566    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="init-config-reloader"
Jul 29 17:55:50.057620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057578    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="alertmanager"
Jul 29 17:55:50.057620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057583    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="alertmanager"
Jul 29 17:55:50.057620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057593    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy"
Jul 29 17:55:50.057620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057601    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy"
Jul 29 17:55:50.057620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057608    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-metric"
Jul 29 17:55:50.057620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057613    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-metric"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057626    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="config-reloader"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057631    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="config-reloader"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:55:50.057632    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976\": container with ID starting with 3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976 not found: ID does not exist" containerID="3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057651    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"} err="failed to get container status \"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976\": rpc error: code = NotFound desc = could not find container \"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976\": container with ID starting with 3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976 not found: ID does not exist"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057665    2560 scope.go:117] "RemoveContainer" containerID="93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057640    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-web"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057707    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-web"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057717    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="prom-label-proxy"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057723    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="prom-label-proxy"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057811    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-web"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057821    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="alertmanager"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057828    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057835    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="kube-rbac-proxy-metric"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057841    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="prom-label-proxy"
Jul 29 17:55:50.057857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057846    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" containerName="config-reloader"
Jul 29 17:55:50.058424 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:55:50.057909    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86\": container with ID starting with 93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86 not found: ID does not exist" containerID="93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"
Jul 29 17:55:50.058424 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057931    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"} err="failed to get container status \"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86\": rpc error: code = NotFound desc = could not find container \"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86\": container with ID starting with 93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86 not found: ID does not exist"
Jul 29 17:55:50.058424 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.057949    2560 scope.go:117] "RemoveContainer" containerID="48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5"
Jul 29 17:55:50.058424 ip-10-0-130-240 kubenswrapper[2560]: E0729 17:55:50.058243    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5\": container with ID starting with 48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5 not found: ID does not exist" containerID="48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5"
Jul 29 17:55:50.058424 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.058290    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5"} err="failed to get container status \"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5\": rpc error: code = NotFound desc = could not find container \"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5\": container with ID starting with 48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5 not found: ID does not exist"
Jul 29 17:55:50.058424 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.058311    2560 scope.go:117] "RemoveContainer" containerID="0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"
Jul 29 17:55:50.058695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.058642    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e"} err="failed to get container status \"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e\": rpc error: code = NotFound desc = could not find container \"0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e\": container with ID starting with 0626e34aeaf055b8df54a653563a94d2dc2282cd0081ae70c84689c7aa7af60e not found: ID does not exist"
Jul 29 17:55:50.058695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.058669    2560 scope.go:117] "RemoveContainer" containerID="55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"
Jul 29 17:55:50.059023 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.058968    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0"} err="failed to get container status \"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0\": rpc error: code = NotFound desc = could not find container \"55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0\": container with ID starting with 55dc8b7a34177f3af1c9c2e7611308f6f62b6acbf342b2568ebee22b02def5e0 not found: ID does not exist"
Jul 29 17:55:50.059023 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.059002    2560 scope.go:117] "RemoveContainer" containerID="2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"
Jul 29 17:55:50.059523 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.059427    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe"} err="failed to get container status \"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe\": rpc error: code = NotFound desc = could not find container \"2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe\": container with ID starting with 2de5256324d34ec38a81109f5ab24ba604806ed10d797af1c79b069664568dbe not found: ID does not exist"
Jul 29 17:55:50.059523 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.059452    2560 scope.go:117] "RemoveContainer" containerID="2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"
Jul 29 17:55:50.059959 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.059869    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18"} err="failed to get container status \"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18\": rpc error: code = NotFound desc = could not find container \"2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18\": container with ID starting with 2815a266cfc25a7b2d02b203ad1885a6e3646e9e66025c8633c8e7d22a5e6e18 not found: ID does not exist"
Jul 29 17:55:50.059959 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.059892    2560 scope.go:117] "RemoveContainer" containerID="3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"
Jul 29 17:55:50.060883 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.060326    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976"} err="failed to get container status \"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976\": rpc error: code = NotFound desc = could not find container \"3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976\": container with ID starting with 3f7057029698d55a7635a07586fb5ed3867230247b7ab7461c1881982803f976 not found: ID does not exist"
Jul 29 17:55:50.060883 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.060353    2560 scope.go:117] "RemoveContainer" containerID="93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"
Jul 29 17:55:50.061343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.061319    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86"} err="failed to get container status \"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86\": rpc error: code = NotFound desc = could not find container \"93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86\": container with ID starting with 93a2a320fc42619707f2421aff3da02157c35e9d39efe59e92cadf7f2bd5ca86 not found: ID does not exist"
Jul 29 17:55:50.061343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.061342    2560 scope.go:117] "RemoveContainer" containerID="48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5"
Jul 29 17:55:50.061655 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.061636    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5"} err="failed to get container status \"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5\": rpc error: code = NotFound desc = could not find container \"48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5\": container with ID starting with 48d93e09a58019e0917ac7956795024050c85456ec1a45e655939b5a86514ab5 not found: ID does not exist"
Jul 29 17:55:50.062810 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.062794    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.064713 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.064677    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 17:55:50.064835 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.064810    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 17:55:50.064835 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.064832    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 17:55:50.064993 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.064835    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 17:55:50.065309 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.065290    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 17:55:50.065309 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.065300    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6fdxf\""
Jul 29 17:55:50.065503 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.065381    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 17:55:50.065503 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.065465    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 17:55:50.065742 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.065723    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 17:55:50.069977 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.069955    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 17:55:50.073113 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.073096    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:55:50.201216 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201130    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201216 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201216 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxvtt\" (UniqueName: \"kubernetes.io/projected/ddf72b70-20e6-4650-8b39-20d2989ad867-kube-api-access-dxvtt\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201475 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201304    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf72b70-20e6-4650-8b39-20d2989ad867-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201475 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201344    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201475 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201475 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf72b70-20e6-4650-8b39-20d2989ad867-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201475 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201450    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf72b70-20e6-4650-8b39-20d2989ad867-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201475 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf72b70-20e6-4650-8b39-20d2989ad867-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201741 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201567    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-web-config\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201741 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201602    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-config-volume\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201741 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.201741 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.201677    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf72b70-20e6-4650-8b39-20d2989ad867-config-out\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.280346 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.280307    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7" path="/var/lib/kubelet/pods/9c2e4292-1dbd-4c13-b97d-e7c24e2f32c7/volumes"
Jul 29 17:55:50.302221 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302221 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302468 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf72b70-20e6-4650-8b39-20d2989ad867-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302468 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf72b70-20e6-4650-8b39-20d2989ad867-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302468 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf72b70-20e6-4650-8b39-20d2989ad867-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302468 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-web-config\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302468 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-config-volume\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302468 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302468 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf72b70-20e6-4650-8b39-20d2989ad867-config-out\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302759 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302477    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302759 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302759 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302542    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxvtt\" (UniqueName: \"kubernetes.io/projected/ddf72b70-20e6-4650-8b39-20d2989ad867-kube-api-access-dxvtt\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302759 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf72b70-20e6-4650-8b39-20d2989ad867-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.302967 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.302883    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ddf72b70-20e6-4650-8b39-20d2989ad867-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.303821 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.303319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf72b70-20e6-4650-8b39-20d2989ad867-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.304842 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.304817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf72b70-20e6-4650-8b39-20d2989ad867-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.305611 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.305588    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.305778 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.305755    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddf72b70-20e6-4650-8b39-20d2989ad867-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.305938 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.305900    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-web-config\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.306075 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.306055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.306129 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.306055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.306129 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.306106    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-config-volume\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.306207 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.306192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddf72b70-20e6-4650-8b39-20d2989ad867-config-out\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.307298 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.307246    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.307374 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.307345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ddf72b70-20e6-4650-8b39-20d2989ad867-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.311017 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.310994    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxvtt\" (UniqueName: \"kubernetes.io/projected/ddf72b70-20e6-4650-8b39-20d2989ad867-kube-api-access-dxvtt\") pod \"alertmanager-main-0\" (UID: \"ddf72b70-20e6-4650-8b39-20d2989ad867\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.374134 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.374097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:55:50.508620 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:50.508585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:55:50.511640 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:55:50.511615    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf72b70_20e6_4650_8b39_20d2989ad867.slice/crio-afe3a5019c1e0b2876dc275dbcaf5be568173191cd19934dfeab11c72aa118b7 WatchSource:0}: Error finding container afe3a5019c1e0b2876dc275dbcaf5be568173191cd19934dfeab11c72aa118b7: Status 404 returned error can't find the container with id afe3a5019c1e0b2876dc275dbcaf5be568173191cd19934dfeab11c72aa118b7
Jul 29 17:55:51.009087 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:51.009054    2560 generic.go:356] "Generic (PLEG): container finished" podID="ddf72b70-20e6-4650-8b39-20d2989ad867" containerID="5fdde7835b2963a53e07365cb7132d7d0901efb18c51b13669a54c0173ee76ff" exitCode=0
Jul 29 17:55:51.009478 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:51.009118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerDied","Data":"5fdde7835b2963a53e07365cb7132d7d0901efb18c51b13669a54c0173ee76ff"}
Jul 29 17:55:51.009478 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:51.009194    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerStarted","Data":"afe3a5019c1e0b2876dc275dbcaf5be568173191cd19934dfeab11c72aa118b7"}
Jul 29 17:55:52.016362 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.016328    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerStarted","Data":"aa7ef61dce5f3d4563b230800c887292b155fd9343da26c9c15b2d3d21912e44"}
Jul 29 17:55:52.016746 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.016407    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerStarted","Data":"97257236d51164d4685517c4ade5676296eb3085e51c9a9abb9eb193cb91be7e"}
Jul 29 17:55:52.016746 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.016434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerStarted","Data":"e3008ea5b74050f989a842a244cf57679399f036775628c065c613a81aba27ff"}
Jul 29 17:55:52.016746 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.016465    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerStarted","Data":"7d3b989a3d5fe4211306086ee4cfeec1fd5704cf918b4db9fc65c13c3270da8c"}
Jul 29 17:55:52.016746 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.016506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerStarted","Data":"028aea574cfb99cde0086f76056e23587cb8c3758054a4f7f7cc8f942e03ef80"}
Jul 29 17:55:52.016746 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.016533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ddf72b70-20e6-4650-8b39-20d2989ad867","Type":"ContainerStarted","Data":"82c9e946cf60464339d1cdf2c51ae74c8330e87fc5c178260e15ae9731dbdbfe"}
Jul 29 17:55:52.041111 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.041053    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.024288881 podStartE2EDuration="2.041034997s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.746116ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:55:52.038353463 +0000 UTC m=+142.280314500" watchObservedRunningTime="2026-07-29 17:55:52.041034997 +0000 UTC m=+142.282996041"
Jul 29 17:55:52.537223 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.537187    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f96947f78-5lzf2"]
Jul 29 17:55:52.539935 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.539917    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.541867 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.541846    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 17:55:52.542258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.542226    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tls4l\""
Jul 29 17:55:52.542412 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.542318    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 17:55:52.542412 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.542401    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 17:55:52.542547 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.542409    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 17:55:52.543162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.543146    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 17:55:52.547160 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.547141    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 17:55:52.556058 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.556020    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f96947f78-5lzf2"]
Jul 29 17:55:52.625134 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.625322 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625148    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-secret-telemeter-client\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.625322 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625211    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-federate-client-tls\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.625322 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-metrics-client-ca\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.625322 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625298    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-telemeter-client-tls\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.625464 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-serving-certs-ca-bundle\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.625464 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625358    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.625464 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.625382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2s65\" (UniqueName: \"kubernetes.io/projected/c1913a48-ecb5-492e-a511-e3e2ec0342b3-kube-api-access-p2s65\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726342 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-serving-certs-ca-bundle\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726500 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726360    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726500 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2s65\" (UniqueName: \"kubernetes.io/projected/c1913a48-ecb5-492e-a511-e3e2ec0342b3-kube-api-access-p2s65\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726500 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726731 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726504    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-secret-telemeter-client\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726731 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-federate-client-tls\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726731 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-metrics-client-ca\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.726731 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.726630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-telemeter-client-tls\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.727161 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.727105    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-serving-certs-ca-bundle\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.727432 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.727413    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.727912 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.727888    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1913a48-ecb5-492e-a511-e3e2ec0342b3-metrics-client-ca\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.729483 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.729435    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.730100 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.730063    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-telemeter-client-tls\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.730186 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.730136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-federate-client-tls\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.731242 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.730624    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c1913a48-ecb5-492e-a511-e3e2ec0342b3-secret-telemeter-client\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.734414 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.734396    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2s65\" (UniqueName: \"kubernetes.io/projected/c1913a48-ecb5-492e-a511-e3e2ec0342b3-kube-api-access-p2s65\") pod \"telemeter-client-7f96947f78-5lzf2\" (UID: \"c1913a48-ecb5-492e-a511-e3e2ec0342b3\") " pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:52.833735 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.833333    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:52.834687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.833956    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="prometheus" containerID="cri-o://95c470dcb39713c90ea4e4b8667c9575fe59eccaffb86c256b27b6ff3ba9ce41" gracePeriod=600
Jul 29 17:55:52.834687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.834207    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2ec4956ade96d5d8ab235e7ecbd2260277e0d65761a8bdcb1db61bc1977ef629" gracePeriod=600
Jul 29 17:55:52.834687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.834332    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy" containerID="cri-o://110ea0d9eec97d026439bfc18d1dfc83574c53ffe20d19b11e6360d29507b54b" gracePeriod=600
Jul 29 17:55:52.834687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.834425    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-web" containerID="cri-o://09f33447760de40c2f7176fc18975b66667d61f215be70546cdac899675d8b00" gracePeriod=600
Jul 29 17:55:52.834687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.834518    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="thanos-sidecar" containerID="cri-o://47715d55f950dcbbfeaba8d757b478f3c4fe4fd090294b352cb8051820a0789f" gracePeriod=600
Jul 29 17:55:52.834687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.834601    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="config-reloader" containerID="cri-o://dea456dea3bd51065e30a80533aef102cb2b421b31d3c5decc78c657cf72153e" gracePeriod=600
Jul 29 17:55:52.853176 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:52.853148    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2"
Jul 29 17:55:53.017384 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.017354    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f96947f78-5lzf2"]
Jul 29 17:55:53.019079 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:55:53.019052    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1913a48_ecb5_492e_a511_e3e2ec0342b3.slice/crio-371c82648e71fe3d646fec6df94192d2995fa5d0f26dedee79290b46212600f3 WatchSource:0}: Error finding container 371c82648e71fe3d646fec6df94192d2995fa5d0f26dedee79290b46212600f3: Status 404 returned error can't find the container with id 371c82648e71fe3d646fec6df94192d2995fa5d0f26dedee79290b46212600f3
Jul 29 17:55:53.025524 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025494    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerID="2ec4956ade96d5d8ab235e7ecbd2260277e0d65761a8bdcb1db61bc1977ef629" exitCode=0
Jul 29 17:55:53.025628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025526    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerID="110ea0d9eec97d026439bfc18d1dfc83574c53ffe20d19b11e6360d29507b54b" exitCode=0
Jul 29 17:55:53.025628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025538    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerID="09f33447760de40c2f7176fc18975b66667d61f215be70546cdac899675d8b00" exitCode=0
Jul 29 17:55:53.025628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025546    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerID="47715d55f950dcbbfeaba8d757b478f3c4fe4fd090294b352cb8051820a0789f" exitCode=0
Jul 29 17:55:53.025628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025555    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerID="dea456dea3bd51065e30a80533aef102cb2b421b31d3c5decc78c657cf72153e" exitCode=0
Jul 29 17:55:53.025628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025563    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerID="95c470dcb39713c90ea4e4b8667c9575fe59eccaffb86c256b27b6ff3ba9ce41" exitCode=0
Jul 29 17:55:53.025628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"2ec4956ade96d5d8ab235e7ecbd2260277e0d65761a8bdcb1db61bc1977ef629"}
Jul 29 17:55:53.025822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"110ea0d9eec97d026439bfc18d1dfc83574c53ffe20d19b11e6360d29507b54b"}
Jul 29 17:55:53.025822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"09f33447760de40c2f7176fc18975b66667d61f215be70546cdac899675d8b00"}
Jul 29 17:55:53.025822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025726    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"47715d55f950dcbbfeaba8d757b478f3c4fe4fd090294b352cb8051820a0789f"}
Jul 29 17:55:53.025822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025762    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"dea456dea3bd51065e30a80533aef102cb2b421b31d3c5decc78c657cf72153e"}
Jul 29 17:55:53.025822 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.025795    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"95c470dcb39713c90ea4e4b8667c9575fe59eccaffb86c256b27b6ff3ba9ce41"}
Jul 29 17:55:53.123896 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.123873    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:53.231516 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231486    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-metrics-client-certs\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.231695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231524    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsf5m\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-kube-api-access-wsf5m\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.231695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231549    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-tls-assets\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.231695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231574    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-metrics-client-ca\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.231695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231601    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-tls\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.231695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231632    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-trusted-ca-bundle\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.231695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231662    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-grpc-tls\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.231695 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231687    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-web-config\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231735    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231763    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231788    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-serving-certs-ca-bundle\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231820    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-kube-rbac-proxy\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231850    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config-out\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231874    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-2\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231902    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-kubelet-serving-ca-bundle\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231929    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-thanos-prometheus-http-client-file\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231972    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-1\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.231999    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.232014    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-db\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232065 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.232043    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:53.232850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.232076    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.232135    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-0\") pod \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\" (UID: \"9a73d81b-162e-4a3a-ab35-9d831e5af1aa\") "
Jul 29 17:55:53.232850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.232460    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-metrics-client-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.232850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.232482    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.233352 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.233254    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:55:53.233789 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.233762    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:53.234763 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.234534    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:53.234763 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.234750    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.235023 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.234391    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:53.235587 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.235178    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:55:53.235587 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.235347    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:53.235587 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.235396    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:53.235587 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.235553    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-kube-api-access-wsf5m" (OuterVolumeSpecName: "kube-api-access-wsf5m") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "kube-api-access-wsf5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:55:53.236408 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.236212    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config-out" (OuterVolumeSpecName: "config-out") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:55:53.236408 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.236404    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.236671 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.236462    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.236671 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.236558    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config" (OuterVolumeSpecName: "config") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.236916 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.236893    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.237009 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.236987    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.237640 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.237622    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.237904 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.237881    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.248036 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.248010    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-web-config" (OuterVolumeSpecName: "web-config") pod "9a73d81b-162e-4a3a-ab35-9d831e5af1aa" (UID: "9a73d81b-162e-4a3a-ab35-9d831e5af1aa"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:53.333343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333298    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333335    2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-grpc-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333344    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-web-config\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333343 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333353    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333363    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333375    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333385    2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-kube-rbac-proxy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333393    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config-out\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333401    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333409    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333418    2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333427    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333435    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-db\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333445    2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-config\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333452    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333461    2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-secret-metrics-client-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333469    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsf5m\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-kube-api-access-wsf5m\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:53.333597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:53.333476    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a73d81b-162e-4a3a-ab35-9d831e5af1aa-tls-assets\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 17:55:54.034510 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.034471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a73d81b-162e-4a3a-ab35-9d831e5af1aa","Type":"ContainerDied","Data":"6c863352674900329c651934912cf186ec02bf0e19c9f68a6f822931fb10b559"}
Jul 29 17:55:54.034510 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.034503    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.035012 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.034578    2560 scope.go:117] "RemoveContainer" containerID="2ec4956ade96d5d8ab235e7ecbd2260277e0d65761a8bdcb1db61bc1977ef629"
Jul 29 17:55:54.036121 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.036088    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2" event={"ID":"c1913a48-ecb5-492e-a511-e3e2ec0342b3","Type":"ContainerStarted","Data":"371c82648e71fe3d646fec6df94192d2995fa5d0f26dedee79290b46212600f3"}
Jul 29 17:55:54.043993 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.043975    2560 scope.go:117] "RemoveContainer" containerID="110ea0d9eec97d026439bfc18d1dfc83574c53ffe20d19b11e6360d29507b54b"
Jul 29 17:55:54.052483 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.052429    2560 scope.go:117] "RemoveContainer" containerID="09f33447760de40c2f7176fc18975b66667d61f215be70546cdac899675d8b00"
Jul 29 17:55:54.059101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.059078    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:54.061632 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.061512    2560 scope.go:117] "RemoveContainer" containerID="47715d55f950dcbbfeaba8d757b478f3c4fe4fd090294b352cb8051820a0789f"
Jul 29 17:55:54.063306 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.063282    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:54.070970 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.070949    2560 scope.go:117] "RemoveContainer" containerID="dea456dea3bd51065e30a80533aef102cb2b421b31d3c5decc78c657cf72153e"
Jul 29 17:55:54.079778 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.079750    2560 scope.go:117] "RemoveContainer" containerID="95c470dcb39713c90ea4e4b8667c9575fe59eccaffb86c256b27b6ff3ba9ce41"
Jul 29 17:55:54.088295 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.087918    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088298    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="thanos-sidecar"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088311    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="thanos-sidecar"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088324    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="prometheus"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088330    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="prometheus"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088339    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="config-reloader"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088344    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="config-reloader"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088355    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="init-config-reloader"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088364    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="init-config-reloader"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088373    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-web"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088380    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-web"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088389    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-thanos"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088398    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-thanos"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088408    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy"
Jul 29 17:55:54.088433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088415    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy"
Jul 29 17:55:54.088972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088474    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="config-reloader"
Jul 29 17:55:54.088972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088483    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-thanos"
Jul 29 17:55:54.088972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088496    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="thanos-sidecar"
Jul 29 17:55:54.088972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088506    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy"
Jul 29 17:55:54.088972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088514    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="kube-rbac-proxy-web"
Jul 29 17:55:54.088972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.088524    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" containerName="prometheus"
Jul 29 17:55:54.090087 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.090061    2560 scope.go:117] "RemoveContainer" containerID="9fa98e1a95db6e27f1514853f6b6327b9d7a01ec28426b78d2fec6889e59e425"
Jul 29 17:55:54.092485 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.092463    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.094514 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.094485    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 17:55:54.094645 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.094563    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 17:55:54.094769 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.094750    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1ponm0rtt4fr3\""
Jul 29 17:55:54.094832 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.094818    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 17:55:54.094893 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.094844    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-skqvf\""
Jul 29 17:55:54.094973 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.094956    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 17:55:54.095067 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095049    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 17:55:54.095129 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095091    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 17:55:54.095185 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095141    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 17:55:54.095185 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095169    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 17:55:54.095326 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095191    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 17:55:54.095430 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095414    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 17:55:54.095485 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095456    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 17:55:54.095861 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.095842    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 17:55:54.098205 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.098042    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 17:55:54.100660 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.100640    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 17:55:54.108879 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.108853    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:54.241560 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241517    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.241560 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241560    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-config\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.241802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241580    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.241802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241615    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.241802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwwcd\" (UniqueName: \"kubernetes.io/projected/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-kube-api-access-qwwcd\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.241802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.241802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.241802 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241853    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241911    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.241996    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.242029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.242048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242101 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.242088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242508 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.242165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242508 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.242192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242508 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.242220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.242508 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.242236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.281359 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.281328    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a73d81b-162e-4a3a-ab35-9d831e5af1aa" path="/var/lib/kubelet/pods/9a73d81b-162e-4a3a-ab35-9d831e5af1aa/volumes"
Jul 29 17:55:54.343025 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.342937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343025 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.342971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-config\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343025 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.342992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343324 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343117    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343324 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343155    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwwcd\" (UniqueName: \"kubernetes.io/projected/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-kube-api-access-qwwcd\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343324 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343206    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343324 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343234    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343324 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343324 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343351    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343421    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343459    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343497    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343527    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343554    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.343652 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.344232 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343657    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.344232 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.344232 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.343718    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.344403 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.344315    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.344820 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.344793    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.347015 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.346091    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.347015 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.346775    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.347166 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.347060    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.347215 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.347191    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.347557 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.347529    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-config\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.349162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.348349    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.349162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.348507    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.349162 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.348583    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.349664 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.349638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.349907 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.349882    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.353082 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.353004    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.353252 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.353220    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.353582 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.353560    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.353708 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.353685    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.354331 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.354260    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.355498 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.355473    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.357383 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.357364    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwwcd\" (UniqueName: \"kubernetes.io/projected/e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa-kube-api-access-qwwcd\") pod \"prometheus-k8s-0\" (UID: \"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.405456 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.405416    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:54.722059 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:54.722037    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:54.734370 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:55:54.734342    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4eeb74e_b120_47b8_bb9e_fa0c7e5486aa.slice/crio-91270891fce1220b7d234116e2d984fa342bfa137e8e7f010c45af4c2a727d38 WatchSource:0}: Error finding container 91270891fce1220b7d234116e2d984fa342bfa137e8e7f010c45af4c2a727d38: Status 404 returned error can't find the container with id 91270891fce1220b7d234116e2d984fa342bfa137e8e7f010c45af4c2a727d38
Jul 29 17:55:55.041837 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:55.041801    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2" event={"ID":"c1913a48-ecb5-492e-a511-e3e2ec0342b3","Type":"ContainerStarted","Data":"9429ad1900ae52ddd807b3b03c51988f14c8d355d99c33a837c1130e4b0a1f35"}
Jul 29 17:55:55.042229 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:55.041868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2" event={"ID":"c1913a48-ecb5-492e-a511-e3e2ec0342b3","Type":"ContainerStarted","Data":"81d2d883edf12ce74dfca637958503ef3a639b4a79770c8fb3f7102f8379c665"}
Jul 29 17:55:55.042229 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:55.041896    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2" event={"ID":"c1913a48-ecb5-492e-a511-e3e2ec0342b3","Type":"ContainerStarted","Data":"2a45e2f7c2cde07f06ad984be16e5ddd219c23866f21a2c147afbf08aab712f9"}
Jul 29 17:55:55.043371 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:55.043345    2560 generic.go:356] "Generic (PLEG): container finished" podID="e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa" containerID="5036807b9efc5bd2e6b216eab7ab436c0e9cc08bcef7302116c22abd7fbd3235" exitCode=0
Jul 29 17:55:55.043501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:55.043433    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerDied","Data":"5036807b9efc5bd2e6b216eab7ab436c0e9cc08bcef7302116c22abd7fbd3235"}
Jul 29 17:55:55.043501 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:55.043494    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerStarted","Data":"91270891fce1220b7d234116e2d984fa342bfa137e8e7f010c45af4c2a727d38"}
Jul 29 17:55:55.060937 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:55.060879    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f96947f78-5lzf2" podStartSLOduration=1.471469846 podStartE2EDuration="3.06085826s" totalImagesPullingTime="1.589388414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:55:55.059139641 +0000 UTC m=+145.301100680" watchObservedRunningTime="2026-07-29 17:55:55.06085826 +0000 UTC m=+145.302819299"
Jul 29 17:55:56.052433 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:56.052398    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerStarted","Data":"209386115b4bc4131d761279064d7e64be1b294ee240dfb31e4e4ba8a0d521c6"}
Jul 29 17:55:56.052850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:56.052470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerStarted","Data":"995dec56c893cb12b14d36c071d558a25be30a416f89f88c0164a15dc1ac7509"}
Jul 29 17:55:56.052850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:56.052500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerStarted","Data":"e49915808af151bd548ef7ab4aea578863d8fa2ea2b993a46ae3629187779fcb"}
Jul 29 17:55:56.052850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:56.052535    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerStarted","Data":"672c2d27191bdc3432526d5fa5fdff56a274ae37804508c654ef51220cd6e160"}
Jul 29 17:55:56.052850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:56.052559    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerStarted","Data":"f4953a4f6d3881ff32690f0d30f0b68e471e7682e6d9e8589e61c32bbffdd003"}
Jul 29 17:55:56.052850 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:56.052582    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa","Type":"ContainerStarted","Data":"f3ff10642424130d0ab80e511a3008250053d400db6b22919b1a512d0b92996c"}
Jul 29 17:55:56.082756 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:56.082704    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.062897613 podStartE2EDuration="2.082689292s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.791679ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:55:56.08196648 +0000 UTC m=+146.323927517" watchObservedRunningTime="2026-07-29 17:55:56.082689292 +0000 UTC m=+146.324650330"
Jul 29 17:55:59.405554 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:55:59.405496    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:56:54.405745 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:56:54.405710    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:56:54.421770 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:56:54.421743    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:56:55.305663 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:56:55.305630    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:58:30.118106 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:30.118075    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 17:58:30.122389 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:30.122370    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 17:58:30.123216 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:30.123200    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 17:58:34.409222 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.409192    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"]
Jul 29 17:58:34.412325 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.412305    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.414687 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.414663    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qsvpq\""
Jul 29 17:58:34.415156 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.415137    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 17:58:34.415241 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.415163    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 17:58:34.429874 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.429851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"]
Jul 29 17:58:34.536114 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.536068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1fd924d-5c81-4f76-8762-c0f7a75e82a2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rt7nk\" (UID: \"c1fd924d-5c81-4f76-8762-c0f7a75e82a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.536114 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.536111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5g9k\" (UniqueName: \"kubernetes.io/projected/c1fd924d-5c81-4f76-8762-c0f7a75e82a2-kube-api-access-w5g9k\") pod \"cert-manager-cainjector-559cdd8c87-rt7nk\" (UID: \"c1fd924d-5c81-4f76-8762-c0f7a75e82a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.637003 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.636964    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1fd924d-5c81-4f76-8762-c0f7a75e82a2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rt7nk\" (UID: \"c1fd924d-5c81-4f76-8762-c0f7a75e82a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.637003 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.637007    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5g9k\" (UniqueName: \"kubernetes.io/projected/c1fd924d-5c81-4f76-8762-c0f7a75e82a2-kube-api-access-w5g9k\") pod \"cert-manager-cainjector-559cdd8c87-rt7nk\" (UID: \"c1fd924d-5c81-4f76-8762-c0f7a75e82a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.644464 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.644434    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1fd924d-5c81-4f76-8762-c0f7a75e82a2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rt7nk\" (UID: \"c1fd924d-5c81-4f76-8762-c0f7a75e82a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.644580 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.644519    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5g9k\" (UniqueName: \"kubernetes.io/projected/c1fd924d-5c81-4f76-8762-c0f7a75e82a2-kube-api-access-w5g9k\") pod \"cert-manager-cainjector-559cdd8c87-rt7nk\" (UID: \"c1fd924d-5c81-4f76-8762-c0f7a75e82a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.729676 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.729589    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"
Jul 29 17:58:34.859739 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.859708    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk"]
Jul 29 17:58:34.863342 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:58:34.863318    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1fd924d_5c81_4f76_8762_c0f7a75e82a2.slice/crio-fa82c6bd881cd2f2d4829c8f45a7895a0c1dad53618b31873727d79c430e68bf WatchSource:0}: Error finding container fa82c6bd881cd2f2d4829c8f45a7895a0c1dad53618b31873727d79c430e68bf: Status 404 returned error can't find the container with id fa82c6bd881cd2f2d4829c8f45a7895a0c1dad53618b31873727d79c430e68bf
Jul 29 17:58:34.865675 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:34.865659    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:58:35.674033 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:35.673999    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk" event={"ID":"c1fd924d-5c81-4f76-8762-c0f7a75e82a2","Type":"ContainerStarted","Data":"fa82c6bd881cd2f2d4829c8f45a7895a0c1dad53618b31873727d79c430e68bf"}
Jul 29 17:58:38.689972 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:38.689936    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk" event={"ID":"c1fd924d-5c81-4f76-8762-c0f7a75e82a2","Type":"ContainerStarted","Data":"6f41179b80c75defae2cd336cc64fd772a0d74eefdc22658ef797aa85b3b01f8"}
Jul 29 17:58:38.705857 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:38.705808    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rt7nk" podStartSLOduration=1.470701639 podStartE2EDuration="4.705791892s" totalImagesPullingTime="3.235090253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:38.703827874 +0000 UTC m=+308.945788914" watchObservedRunningTime="2026-07-29 17:58:38.705791892 +0000 UTC m=+308.947752929"
Jul 29 17:58:43.595660 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.595623    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6d44g"]
Jul 29 17:58:43.598935 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.598912    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:43.600959 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.600938    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-w4qq9\""
Jul 29 17:58:43.611146 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.611121    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6d44g"]
Jul 29 17:58:43.717608 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.717572    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcp2h\" (UniqueName: \"kubernetes.io/projected/434275a9-3a63-4c08-aeec-6da4ec680611-kube-api-access-zcp2h\") pod \"cert-manager-55b4d954d5-6d44g\" (UID: \"434275a9-3a63-4c08-aeec-6da4ec680611\") " pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:43.717778 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.717684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/434275a9-3a63-4c08-aeec-6da4ec680611-bound-sa-token\") pod \"cert-manager-55b4d954d5-6d44g\" (UID: \"434275a9-3a63-4c08-aeec-6da4ec680611\") " pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:43.819155 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.819115    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcp2h\" (UniqueName: \"kubernetes.io/projected/434275a9-3a63-4c08-aeec-6da4ec680611-kube-api-access-zcp2h\") pod \"cert-manager-55b4d954d5-6d44g\" (UID: \"434275a9-3a63-4c08-aeec-6da4ec680611\") " pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:43.819357 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.819217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/434275a9-3a63-4c08-aeec-6da4ec680611-bound-sa-token\") pod \"cert-manager-55b4d954d5-6d44g\" (UID: \"434275a9-3a63-4c08-aeec-6da4ec680611\") " pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:43.827320 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.827259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/434275a9-3a63-4c08-aeec-6da4ec680611-bound-sa-token\") pod \"cert-manager-55b4d954d5-6d44g\" (UID: \"434275a9-3a63-4c08-aeec-6da4ec680611\") " pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:43.827575 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.827555    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcp2h\" (UniqueName: \"kubernetes.io/projected/434275a9-3a63-4c08-aeec-6da4ec680611-kube-api-access-zcp2h\") pod \"cert-manager-55b4d954d5-6d44g\" (UID: \"434275a9-3a63-4c08-aeec-6da4ec680611\") " pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:43.907663 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:43.907578    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6d44g"
Jul 29 17:58:44.035387 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:44.035360    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6d44g"]
Jul 29 17:58:44.038363 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:58:44.038336    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod434275a9_3a63_4c08_aeec_6da4ec680611.slice/crio-698e5293b6696e72ea0838878a6751f0c4c60785f8712e8e1297017281da8e42 WatchSource:0}: Error finding container 698e5293b6696e72ea0838878a6751f0c4c60785f8712e8e1297017281da8e42: Status 404 returned error can't find the container with id 698e5293b6696e72ea0838878a6751f0c4c60785f8712e8e1297017281da8e42
Jul 29 17:58:44.718218 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:44.718187    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6d44g" event={"ID":"434275a9-3a63-4c08-aeec-6da4ec680611","Type":"ContainerStarted","Data":"b0e455e9b871ef9c2ac530e150d1c55ffdccf4345eded753dec326531dcde029"}
Jul 29 17:58:44.718623 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:44.718237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6d44g" event={"ID":"434275a9-3a63-4c08-aeec-6da4ec680611","Type":"ContainerStarted","Data":"698e5293b6696e72ea0838878a6751f0c4c60785f8712e8e1297017281da8e42"}
Jul 29 17:58:44.732624 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:58:44.732573    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6d44g" podStartSLOduration=1.732557195 podStartE2EDuration="1.732557195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:44.731480378 +0000 UTC m=+314.973441416" watchObservedRunningTime="2026-07-29 17:58:44.732557195 +0000 UTC m=+314.974518233"
Jul 29 17:59:04.258559 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.258524    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"]
Jul 29 17:59:04.265257 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.265236    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.267470 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.267449    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 17:59:04.268349 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.268329    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wwfrm\""
Jul 29 17:59:04.268450 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.268424    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 17:59:04.268514 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.268484    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 17:59:04.268514 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.268484    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 17:59:04.283798 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.283767    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"]
Jul 29 17:59:04.302449 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.302409    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/614b29cb-83fb-4fcd-824c-7d051959b106-tmp\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.302628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.302530    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614b29cb-83fb-4fcd-824c-7d051959b106-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.302628 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.302574    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/614b29cb-83fb-4fcd-824c-7d051959b106-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.302726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.302638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/614b29cb-83fb-4fcd-824c-7d051959b106-manifests\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.302726 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.302683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wbw6\" (UniqueName: \"kubernetes.io/projected/614b29cb-83fb-4fcd-824c-7d051959b106-kube-api-access-9wbw6\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.403058 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.403023    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/614b29cb-83fb-4fcd-824c-7d051959b106-manifests\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.403255 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.403067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wbw6\" (UniqueName: \"kubernetes.io/projected/614b29cb-83fb-4fcd-824c-7d051959b106-kube-api-access-9wbw6\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.403255 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.403103    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/614b29cb-83fb-4fcd-824c-7d051959b106-tmp\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.403255 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.403151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614b29cb-83fb-4fcd-824c-7d051959b106-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.403255 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.403174    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/614b29cb-83fb-4fcd-824c-7d051959b106-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.403544 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.403510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/614b29cb-83fb-4fcd-824c-7d051959b106-manifests\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.405623 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.405596    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/614b29cb-83fb-4fcd-824c-7d051959b106-tmp\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.405763 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.405743    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/614b29cb-83fb-4fcd-824c-7d051959b106-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.405823 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.405782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614b29cb-83fb-4fcd-824c-7d051959b106-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.414490 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.414459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wbw6\" (UniqueName: \"kubernetes.io/projected/614b29cb-83fb-4fcd-824c-7d051959b106-kube-api-access-9wbw6\") pod \"opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl\" (UID: \"614b29cb-83fb-4fcd-824c-7d051959b106\") " pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.576714 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.576620    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:04.717325 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.717300    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"]
Jul 29 17:59:04.719474 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:59:04.719446    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614b29cb_83fb_4fcd_824c_7d051959b106.slice/crio-1bc724aecd4acd84e2808fa062f47efdda97a6c34a21b353a71057aced54fcb5 WatchSource:0}: Error finding container 1bc724aecd4acd84e2808fa062f47efdda97a6c34a21b353a71057aced54fcb5: Status 404 returned error can't find the container with id 1bc724aecd4acd84e2808fa062f47efdda97a6c34a21b353a71057aced54fcb5
Jul 29 17:59:04.797758 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:04.797726    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl" event={"ID":"614b29cb-83fb-4fcd-824c-7d051959b106","Type":"ContainerStarted","Data":"1bc724aecd4acd84e2808fa062f47efdda97a6c34a21b353a71057aced54fcb5"}
Jul 29 17:59:07.814211 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:07.814169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl" event={"ID":"614b29cb-83fb-4fcd-824c-7d051959b106","Type":"ContainerStarted","Data":"91feef20bf79aa25ce8951e04714cd3f17277343990c74e1fb18aca5f90add73"}
Jul 29 17:59:09.827876 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:09.827840    2560 generic.go:356] "Generic (PLEG): container finished" podID="614b29cb-83fb-4fcd-824c-7d051959b106" containerID="91feef20bf79aa25ce8951e04714cd3f17277343990c74e1fb18aca5f90add73" exitCode=0
Jul 29 17:59:09.828258 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:09.827918    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl" event={"ID":"614b29cb-83fb-4fcd-824c-7d051959b106","Type":"ContainerDied","Data":"91feef20bf79aa25ce8951e04714cd3f17277343990c74e1fb18aca5f90add73"}
Jul 29 17:59:10.839288 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:10.839224    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl" event={"ID":"614b29cb-83fb-4fcd-824c-7d051959b106","Type":"ContainerStarted","Data":"6c7ed0ffa5eae398369980c7199b746ae8ab1bf72a6146f59ecdec8627a60294"}
Jul 29 17:59:10.839679 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:10.839501    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:10.859940 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:10.859890    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl" podStartSLOduration=2.500531612 podStartE2EDuration="6.859874074s" totalImagesPullingTime="3.290007265s" totalInitContainerRuntime="1.069335197s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:59:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:10.857642221 +0000 UTC m=+341.099603260" watchObservedRunningTime="2026-07-29 17:59:10.859874074 +0000 UTC m=+341.101835114"
Jul 29 17:59:21.846819 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:21.846739    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl"
Jul 29 17:59:32.293448 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.293408    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"]
Jul 29 17:59:32.298281 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.298244    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.300555 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.300532    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 17:59:32.300686 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.300535    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 17:59:32.300876 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.300859    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 17:59:32.300954 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.300897    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 17:59:32.300954 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.300921    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n9zcn\""
Jul 29 17:59:32.301177 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.301162    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:59:32.307223 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.307202    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"]
Jul 29 17:59:32.460756 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.460727    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90ce4563-ded9-4246-97e4-eb12b8e91850-manager-config\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.460756 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.460759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90ce4563-ded9-4246-97e4-eb12b8e91850-metrics-cert\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.460958 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.460781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25mmq\" (UniqueName: \"kubernetes.io/projected/90ce4563-ded9-4246-97e4-eb12b8e91850-kube-api-access-25mmq\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.460958 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.460926    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90ce4563-ded9-4246-97e4-eb12b8e91850-cert\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.562049 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.561961    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90ce4563-ded9-4246-97e4-eb12b8e91850-manager-config\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.562049 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.561995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90ce4563-ded9-4246-97e4-eb12b8e91850-metrics-cert\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.562049 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.562010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25mmq\" (UniqueName: \"kubernetes.io/projected/90ce4563-ded9-4246-97e4-eb12b8e91850-kube-api-access-25mmq\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.562339 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.562087    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90ce4563-ded9-4246-97e4-eb12b8e91850-cert\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.562597 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.562577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90ce4563-ded9-4246-97e4-eb12b8e91850-manager-config\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.564709 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.564686    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90ce4563-ded9-4246-97e4-eb12b8e91850-cert\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.564798 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.564754    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90ce4563-ded9-4246-97e4-eb12b8e91850-metrics-cert\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.570066 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.570046    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25mmq\" (UniqueName: \"kubernetes.io/projected/90ce4563-ded9-4246-97e4-eb12b8e91850-kube-api-access-25mmq\") pod \"lws-controller-manager-8569878f48-bb6zg\" (UID: \"90ce4563-ded9-4246-97e4-eb12b8e91850\") " pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.608306 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.608248    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:32.738422 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.738398    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"]
Jul 29 17:59:32.740581 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:59:32.740560    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ce4563_ded9_4246_97e4_eb12b8e91850.slice/crio-3121dd648656f3d1528b8c94926791380aa976bde4cfb464bfa2fe32bbcdfaec WatchSource:0}: Error finding container 3121dd648656f3d1528b8c94926791380aa976bde4cfb464bfa2fe32bbcdfaec: Status 404 returned error can't find the container with id 3121dd648656f3d1528b8c94926791380aa976bde4cfb464bfa2fe32bbcdfaec
Jul 29 17:59:32.937577 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:32.937490    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg" event={"ID":"90ce4563-ded9-4246-97e4-eb12b8e91850","Type":"ContainerStarted","Data":"3121dd648656f3d1528b8c94926791380aa976bde4cfb464bfa2fe32bbcdfaec"}
Jul 29 17:59:34.707234 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.707203    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"]
Jul 29 17:59:34.719732 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.719700    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.723368 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.723337    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 17:59:34.723879 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.723851    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-49xrc\""
Jul 29 17:59:34.724018 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.723850    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 17:59:34.724018 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.723951    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 17:59:34.724330 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.724303    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 17:59:34.727362 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.727339    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"]
Jul 29 17:59:34.885219 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.885176    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svn78\" (UniqueName: \"kubernetes.io/projected/94bf186c-183c-4f7d-98f1-d8d787da0f82-kube-api-access-svn78\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.885414 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.885246    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bf186c-183c-4f7d-98f1-d8d787da0f82-tls-certs\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.885414 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.885378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94bf186c-183c-4f7d-98f1-d8d787da0f82-tmp\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.986728 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.986688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94bf186c-183c-4f7d-98f1-d8d787da0f82-tmp\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.986906 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.986785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svn78\" (UniqueName: \"kubernetes.io/projected/94bf186c-183c-4f7d-98f1-d8d787da0f82-kube-api-access-svn78\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.986906 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.986835    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bf186c-183c-4f7d-98f1-d8d787da0f82-tls-certs\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.989294 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.989256    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94bf186c-183c-4f7d-98f1-d8d787da0f82-tmp\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.989640 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.989621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94bf186c-183c-4f7d-98f1-d8d787da0f82-tls-certs\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:34.994994 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:34.994972    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svn78\" (UniqueName: \"kubernetes.io/projected/94bf186c-183c-4f7d-98f1-d8d787da0f82-kube-api-access-svn78\") pod \"kube-auth-proxy-786b455d9f-c27kd\" (UID: \"94bf186c-183c-4f7d-98f1-d8d787da0f82\") " pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:35.033152 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:35.033117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"
Jul 29 17:59:35.280160 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:35.280137    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-786b455d9f-c27kd"]
Jul 29 17:59:35.281709 ip-10-0-130-240 kubenswrapper[2560]: W0729 17:59:35.281684    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94bf186c_183c_4f7d_98f1_d8d787da0f82.slice/crio-a1b0987c0ac6d9b1bc460f1f9256a2b3a2a8f55f8d9159dc2139bcb40e1dbb18 WatchSource:0}: Error finding container a1b0987c0ac6d9b1bc460f1f9256a2b3a2a8f55f8d9159dc2139bcb40e1dbb18: Status 404 returned error can't find the container with id a1b0987c0ac6d9b1bc460f1f9256a2b3a2a8f55f8d9159dc2139bcb40e1dbb18
Jul 29 17:59:35.953598 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:35.953556    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd" event={"ID":"94bf186c-183c-4f7d-98f1-d8d787da0f82","Type":"ContainerStarted","Data":"a1b0987c0ac6d9b1bc460f1f9256a2b3a2a8f55f8d9159dc2139bcb40e1dbb18"}
Jul 29 17:59:35.955229 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:35.955196    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg" event={"ID":"90ce4563-ded9-4246-97e4-eb12b8e91850","Type":"ContainerStarted","Data":"4b38f964fdff64c2ea6a0381e26997fdd27aab4b83fef0a69af34c7d0d657cc6"}
Jul 29 17:59:35.955408 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:35.955332    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 17:59:35.977730 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:35.977390    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg" podStartSLOduration=1.545261879 podStartE2EDuration="3.97736853s" totalImagesPullingTime="2.432106651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:35.977129668 +0000 UTC m=+366.219090706" watchObservedRunningTime="2026-07-29 17:59:35.97736853 +0000 UTC m=+366.219329569"
Jul 29 17:59:37.969093 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:37.969053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd" event={"ID":"94bf186c-183c-4f7d-98f1-d8d787da0f82","Type":"ContainerStarted","Data":"b319faec5a4187f30f11ee6c6904a0857541b6fc2ff0f08d7d275eb666e95352"}
Jul 29 17:59:37.986385 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:37.986327    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-786b455d9f-c27kd" podStartSLOduration=1.397762723 podStartE2EDuration="3.986308736s" totalImagesPullingTime="2.588546013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:37.986150437 +0000 UTC m=+368.228111476" watchObservedRunningTime="2026-07-29 17:59:37.986308736 +0000 UTC m=+368.228269788"
Jul 29 17:59:46.965617 ip-10-0-130-240 kubenswrapper[2560]: I0729 17:59:46.965589    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8569878f48-bb6zg"
Jul 29 18:01:10.792997 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.792961    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"]
Jul 29 18:01:10.796885 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.796864    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:10.799047 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.799020    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 18:01:10.799287 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.799260    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 18:01:10.799380 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.799360    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 18:01:10.799425 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.799368    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 18:01:10.799768 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.799754    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mxn6k\""
Jul 29 18:01:10.803639 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.803596    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"]
Jul 29 18:01:10.948966 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.948928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a70595f-218a-49a2-96a3-5adee5fdfb57-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:10.949148 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.948971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4a70595f-218a-49a2-96a3-5adee5fdfb57-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:10.949148 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:10.949003    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmnsh\" (UniqueName: \"kubernetes.io/projected/4a70595f-218a-49a2-96a3-5adee5fdfb57-kube-api-access-gmnsh\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.050546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.050455    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a70595f-218a-49a2-96a3-5adee5fdfb57-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.050546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.050493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4a70595f-218a-49a2-96a3-5adee5fdfb57-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.050546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.050516    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmnsh\" (UniqueName: \"kubernetes.io/projected/4a70595f-218a-49a2-96a3-5adee5fdfb57-kube-api-access-gmnsh\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.050809 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:01:11.050625    2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 29 18:01:11.050809 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:01:11.050702    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a70595f-218a-49a2-96a3-5adee5fdfb57-plugin-serving-cert podName:4a70595f-218a-49a2-96a3-5adee5fdfb57 nodeName:}" failed. No retries permitted until 2026-07-29 18:01:11.550680572 +0000 UTC m=+461.792641588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/4a70595f-218a-49a2-96a3-5adee5fdfb57-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-tjqdn" (UID: "4a70595f-218a-49a2-96a3-5adee5fdfb57") : secret "plugin-serving-cert" not found
Jul 29 18:01:11.051138 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.051118    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4a70595f-218a-49a2-96a3-5adee5fdfb57-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.058588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.058565    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmnsh\" (UniqueName: \"kubernetes.io/projected/4a70595f-218a-49a2-96a3-5adee5fdfb57-kube-api-access-gmnsh\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.554373 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.554329    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a70595f-218a-49a2-96a3-5adee5fdfb57-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.556844 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.556814    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a70595f-218a-49a2-96a3-5adee5fdfb57-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tjqdn\" (UID: \"4a70595f-218a-49a2-96a3-5adee5fdfb57\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.707802 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.707765    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"
Jul 29 18:01:11.838767 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:11.838742    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn"]
Jul 29 18:01:11.840748 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:01:11.840723    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a70595f_218a_49a2_96a3_5adee5fdfb57.slice/crio-83337bb3ca52ff1dc4b32832c0ef5b29e545ddcfef22366a141798810568e1af WatchSource:0}: Error finding container 83337bb3ca52ff1dc4b32832c0ef5b29e545ddcfef22366a141798810568e1af: Status 404 returned error can't find the container with id 83337bb3ca52ff1dc4b32832c0ef5b29e545ddcfef22366a141798810568e1af
Jul 29 18:01:12.404231 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:12.404191    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn" event={"ID":"4a70595f-218a-49a2-96a3-5adee5fdfb57","Type":"ContainerStarted","Data":"83337bb3ca52ff1dc4b32832c0ef5b29e545ddcfef22366a141798810568e1af"}
Jul 29 18:01:20.988305 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:20.988253    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"]
Jul 29 18:01:20.999515 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:20.999482    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:21.002159 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.001912    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vznvv\""
Jul 29 18:01:21.012537 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.012503    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"]
Jul 29 18:01:21.042877 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.042708    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqzh8\" (UniqueName: \"kubernetes.io/projected/c8e17ded-bd28-4e3f-8860-02da50876d53-kube-api-access-sqzh8\") pod \"kuadrant-operator-controller-manager-55467d9f7-vxf2n\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:21.042877 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.042785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8e17ded-bd28-4e3f-8860-02da50876d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-vxf2n\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:21.143950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.143912    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqzh8\" (UniqueName: \"kubernetes.io/projected/c8e17ded-bd28-4e3f-8860-02da50876d53-kube-api-access-sqzh8\") pod \"kuadrant-operator-controller-manager-55467d9f7-vxf2n\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:21.144138 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.143985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8e17ded-bd28-4e3f-8860-02da50876d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-vxf2n\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:21.144455 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.144435    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8e17ded-bd28-4e3f-8860-02da50876d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-vxf2n\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:21.155186 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.155156    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqzh8\" (UniqueName: \"kubernetes.io/projected/c8e17ded-bd28-4e3f-8860-02da50876d53-kube-api-access-sqzh8\") pod \"kuadrant-operator-controller-manager-55467d9f7-vxf2n\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:21.314959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:21.314874    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:32.621187 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:32.621161    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"]
Jul 29 18:01:32.623325 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:01:32.623253    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e17ded_bd28_4e3f_8860_02da50876d53.slice/crio-6b31781ec3ea718d73b530139b0e407fa1626f543aa42a2f56b280ec792e98e1 WatchSource:0}: Error finding container 6b31781ec3ea718d73b530139b0e407fa1626f543aa42a2f56b280ec792e98e1: Status 404 returned error can't find the container with id 6b31781ec3ea718d73b530139b0e407fa1626f543aa42a2f56b280ec792e98e1
Jul 29 18:01:33.531580 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:33.531543    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn" event={"ID":"4a70595f-218a-49a2-96a3-5adee5fdfb57","Type":"ContainerStarted","Data":"4877407e42703ec6c461766a9388d61659fe27f4af876b433dd35f212df6a0cc"}
Jul 29 18:01:33.532792 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:33.532767    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n" event={"ID":"c8e17ded-bd28-4e3f-8860-02da50876d53","Type":"ContainerStarted","Data":"6b31781ec3ea718d73b530139b0e407fa1626f543aa42a2f56b280ec792e98e1"}
Jul 29 18:01:33.547617 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:33.547568    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tjqdn" podStartSLOduration=2.796823459 podStartE2EDuration="23.547553336s" totalImagesPullingTime="20.750729877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:33.546466776 +0000 UTC m=+483.788427805" watchObservedRunningTime="2026-07-29 18:01:33.547553336 +0000 UTC m=+483.789514374"
Jul 29 18:01:39.568874 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:39.568834    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n" event={"ID":"c8e17ded-bd28-4e3f-8860-02da50876d53","Type":"ContainerStarted","Data":"050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071"}
Jul 29 18:01:39.569364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:39.568993    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:01:39.588147 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:39.588091    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n" podStartSLOduration=13.35809845 podStartE2EDuration="19.588071837s" totalImagesPullingTime="6.229973387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:39.586555161 +0000 UTC m=+489.828516199" watchObservedRunningTime="2026-07-29 18:01:39.588071837 +0000 UTC m=+489.830032876"
Jul 29 18:01:50.575945 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:01:50.575914    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:02:03.173642 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.173605    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"]
Jul 29 18:02:03.174232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.173798    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n" podUID="c8e17ded-bd28-4e3f-8860-02da50876d53" containerName="manager" containerID="cri-o://050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071" gracePeriod=10
Jul 29 18:02:03.469448 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.469378    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:02:03.533615 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.533585    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8e17ded-bd28-4e3f-8860-02da50876d53-extensions-socket-volume\") pod \"c8e17ded-bd28-4e3f-8860-02da50876d53\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") "
Jul 29 18:02:03.533770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.533625    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqzh8\" (UniqueName: \"kubernetes.io/projected/c8e17ded-bd28-4e3f-8860-02da50876d53-kube-api-access-sqzh8\") pod \"c8e17ded-bd28-4e3f-8860-02da50876d53\" (UID: \"c8e17ded-bd28-4e3f-8860-02da50876d53\") "
Jul 29 18:02:03.534130 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.534098    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e17ded-bd28-4e3f-8860-02da50876d53-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8e17ded-bd28-4e3f-8860-02da50876d53" (UID: "c8e17ded-bd28-4e3f-8860-02da50876d53"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:02:03.535992 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.535963    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8e17ded-bd28-4e3f-8860-02da50876d53-kube-api-access-sqzh8" (OuterVolumeSpecName: "kube-api-access-sqzh8") pod "c8e17ded-bd28-4e3f-8860-02da50876d53" (UID: "c8e17ded-bd28-4e3f-8860-02da50876d53"). InnerVolumeSpecName "kube-api-access-sqzh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:02:03.634829 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.634790    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8e17ded-bd28-4e3f-8860-02da50876d53-extensions-socket-volume\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:03.634829 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.634825    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqzh8\" (UniqueName: \"kubernetes.io/projected/c8e17ded-bd28-4e3f-8860-02da50876d53-kube-api-access-sqzh8\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:03.680976 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.680941    2560 generic.go:356] "Generic (PLEG): container finished" podID="c8e17ded-bd28-4e3f-8860-02da50876d53" containerID="050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071" exitCode=0
Jul 29 18:02:03.681121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.680988    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"
Jul 29 18:02:03.681121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.680990    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n" event={"ID":"c8e17ded-bd28-4e3f-8860-02da50876d53","Type":"ContainerDied","Data":"050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071"}
Jul 29 18:02:03.681121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.681043    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n" event={"ID":"c8e17ded-bd28-4e3f-8860-02da50876d53","Type":"ContainerDied","Data":"6b31781ec3ea718d73b530139b0e407fa1626f543aa42a2f56b280ec792e98e1"}
Jul 29 18:02:03.681121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.681073    2560 scope.go:117] "RemoveContainer" containerID="050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071"
Jul 29 18:02:03.689850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.689834    2560 scope.go:117] "RemoveContainer" containerID="050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071"
Jul 29 18:02:03.690111 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:02:03.690092    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071\": container with ID starting with 050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071 not found: ID does not exist" containerID="050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071"
Jul 29 18:02:03.690167 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.690122    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071"} err="failed to get container status \"050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071\": rpc error: code = NotFound desc = could not find container \"050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071\": container with ID starting with 050caffd862bcf99968f7c83c77e859be36a49de2cb6db693335297f6ab4c071 not found: ID does not exist"
Jul 29 18:02:03.704939 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.704908    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"]
Jul 29 18:02:03.710233 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:03.710210    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-vxf2n"]
Jul 29 18:02:04.281191 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:04.281157    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8e17ded-bd28-4e3f-8860-02da50876d53" path="/var/lib/kubelet/pods/c8e17ded-bd28-4e3f-8860-02da50876d53/volumes"
Jul 29 18:02:12.997489 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:12.997458    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"]
Jul 29 18:02:12.998132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:12.997828    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8e17ded-bd28-4e3f-8860-02da50876d53" containerName="manager"
Jul 29 18:02:12.998132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:12.997839    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e17ded-bd28-4e3f-8860-02da50876d53" containerName="manager"
Jul 29 18:02:12.998132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:12.997914    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8e17ded-bd28-4e3f-8860-02da50876d53" containerName="manager"
Jul 29 18:02:13.018625 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.018600    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"]
Jul 29 18:02:13.018771 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.018715    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.020605 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.020569    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 18:02:13.091722 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.091689    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"]
Jul 29 18:02:13.123343 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.123307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k6nz\" (UniqueName: \"kubernetes.io/projected/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-kube-api-access-8k6nz\") pod \"limitador-limitador-64c94fd64f-wxlbx\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.123485 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.123358    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-config-file\") pod \"limitador-limitador-64c94fd64f-wxlbx\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.224632 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.224593    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k6nz\" (UniqueName: \"kubernetes.io/projected/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-kube-api-access-8k6nz\") pod \"limitador-limitador-64c94fd64f-wxlbx\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.224798 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.224641    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-config-file\") pod \"limitador-limitador-64c94fd64f-wxlbx\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.225208 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.225190    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-config-file\") pod \"limitador-limitador-64c94fd64f-wxlbx\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.232620 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.232596    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k6nz\" (UniqueName: \"kubernetes.io/projected/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-kube-api-access-8k6nz\") pod \"limitador-limitador-64c94fd64f-wxlbx\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.329063 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.328982    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:13.459204 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.459174    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"]
Jul 29 18:02:13.462071 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:02:13.462042    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb53c38_c4ec_4afb_85c9_c28dda3b2360.slice/crio-63751ac70da13592f9e23219cd484fc9cddbd8f4d5c22addf87ee659fc6a1013 WatchSource:0}: Error finding container 63751ac70da13592f9e23219cd484fc9cddbd8f4d5c22addf87ee659fc6a1013: Status 404 returned error can't find the container with id 63751ac70da13592f9e23219cd484fc9cddbd8f4d5c22addf87ee659fc6a1013
Jul 29 18:02:13.727132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.727053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx" event={"ID":"7fb53c38-c4ec-4afb-85c9-c28dda3b2360","Type":"ContainerStarted","Data":"63751ac70da13592f9e23219cd484fc9cddbd8f4d5c22addf87ee659fc6a1013"}
Jul 29 18:02:13.957101 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.957067    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wxfdv"]
Jul 29 18:02:13.960091 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.960075    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wxfdv"
Jul 29 18:02:13.961732 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.961716    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4w7sd\""
Jul 29 18:02:13.965925 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:13.965896    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wxfdv"]
Jul 29 18:02:14.032628 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:14.032589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp9kd\" (UniqueName: \"kubernetes.io/projected/b10daf60-4de2-41fd-a962-ef31c7ddc689-kube-api-access-fp9kd\") pod \"authorino-75d556bd77-wxfdv\" (UID: \"b10daf60-4de2-41fd-a962-ef31c7ddc689\") " pod="kuadrant-system/authorino-75d556bd77-wxfdv"
Jul 29 18:02:14.133259 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:14.133227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp9kd\" (UniqueName: \"kubernetes.io/projected/b10daf60-4de2-41fd-a962-ef31c7ddc689-kube-api-access-fp9kd\") pod \"authorino-75d556bd77-wxfdv\" (UID: \"b10daf60-4de2-41fd-a962-ef31c7ddc689\") " pod="kuadrant-system/authorino-75d556bd77-wxfdv"
Jul 29 18:02:14.141407 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:14.141382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp9kd\" (UniqueName: \"kubernetes.io/projected/b10daf60-4de2-41fd-a962-ef31c7ddc689-kube-api-access-fp9kd\") pod \"authorino-75d556bd77-wxfdv\" (UID: \"b10daf60-4de2-41fd-a962-ef31c7ddc689\") " pod="kuadrant-system/authorino-75d556bd77-wxfdv"
Jul 29 18:02:14.270526 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:14.270489    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wxfdv"
Jul 29 18:02:14.398208 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:14.398179    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wxfdv"]
Jul 29 18:02:14.401254 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:02:14.401221    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb10daf60_4de2_41fd_a962_ef31c7ddc689.slice/crio-270c140f56fcc12d25faec405c47c5892ee3241a514d6ccfd282ebe2a3a26e86 WatchSource:0}: Error finding container 270c140f56fcc12d25faec405c47c5892ee3241a514d6ccfd282ebe2a3a26e86: Status 404 returned error can't find the container with id 270c140f56fcc12d25faec405c47c5892ee3241a514d6ccfd282ebe2a3a26e86
Jul 29 18:02:14.735668 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:14.735563    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wxfdv" event={"ID":"b10daf60-4de2-41fd-a962-ef31c7ddc689","Type":"ContainerStarted","Data":"270c140f56fcc12d25faec405c47c5892ee3241a514d6ccfd282ebe2a3a26e86"}
Jul 29 18:02:18.756447 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:18.756415    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx" event={"ID":"7fb53c38-c4ec-4afb-85c9-c28dda3b2360","Type":"ContainerStarted","Data":"0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b"}
Jul 29 18:02:18.756940 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:18.756497    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:18.758026 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:18.757997    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wxfdv" event={"ID":"b10daf60-4de2-41fd-a962-ef31c7ddc689","Type":"ContainerStarted","Data":"f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632"}
Jul 29 18:02:18.773555 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:18.773504    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx" podStartSLOduration=1.838437333 podStartE2EDuration="6.773489176s" totalImagesPullingTime="4.935051843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:18.771740208 +0000 UTC m=+529.013701246" watchObservedRunningTime="2026-07-29 18:02:18.773489176 +0000 UTC m=+529.015450214"
Jul 29 18:02:18.786251 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:18.786207    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wxfdv" podStartSLOduration=1.788812941 podStartE2EDuration="5.786192578s" totalImagesPullingTime="3.997379637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:18.784709616 +0000 UTC m=+529.026670654" watchObservedRunningTime="2026-07-29 18:02:18.786192578 +0000 UTC m=+529.028153615"
Jul 29 18:02:29.634092 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:29.634017    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"]
Jul 29 18:02:29.634519 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:29.634217    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx" podUID="7fb53c38-c4ec-4afb-85c9-c28dda3b2360" containerName="limitador" containerID="cri-o://0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b" gracePeriod=30
Jul 29 18:02:29.634879 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:29.634860    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:30.182942 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.182918    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:30.286286 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.286239    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-config-file\") pod \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") "
Jul 29 18:02:30.286460 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.286365    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k6nz\" (UniqueName: \"kubernetes.io/projected/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-kube-api-access-8k6nz\") pod \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\" (UID: \"7fb53c38-c4ec-4afb-85c9-c28dda3b2360\") "
Jul 29 18:02:30.286598 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.286571    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-config-file" (OuterVolumeSpecName: "config-file") pod "7fb53c38-c4ec-4afb-85c9-c28dda3b2360" (UID: "7fb53c38-c4ec-4afb-85c9-c28dda3b2360"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:02:30.288356 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.288332    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-kube-api-access-8k6nz" (OuterVolumeSpecName: "kube-api-access-8k6nz") pod "7fb53c38-c4ec-4afb-85c9-c28dda3b2360" (UID: "7fb53c38-c4ec-4afb-85c9-c28dda3b2360"). InnerVolumeSpecName "kube-api-access-8k6nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:02:30.387448 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.387408    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-config-file\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:30.387448 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.387439    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k6nz\" (UniqueName: \"kubernetes.io/projected/7fb53c38-c4ec-4afb-85c9-c28dda3b2360-kube-api-access-8k6nz\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:30.815660 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.815623    2560 generic.go:356] "Generic (PLEG): container finished" podID="7fb53c38-c4ec-4afb-85c9-c28dda3b2360" containerID="0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b" exitCode=0
Jul 29 18:02:30.816114 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.815669    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"
Jul 29 18:02:30.816114 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.815693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx" event={"ID":"7fb53c38-c4ec-4afb-85c9-c28dda3b2360","Type":"ContainerDied","Data":"0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b"}
Jul 29 18:02:30.816114 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.815736    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wxlbx" event={"ID":"7fb53c38-c4ec-4afb-85c9-c28dda3b2360","Type":"ContainerDied","Data":"63751ac70da13592f9e23219cd484fc9cddbd8f4d5c22addf87ee659fc6a1013"}
Jul 29 18:02:30.816114 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.815757    2560 scope.go:117] "RemoveContainer" containerID="0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b"
Jul 29 18:02:30.824751 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.824735    2560 scope.go:117] "RemoveContainer" containerID="0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b"
Jul 29 18:02:30.825014 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:02:30.824996    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b\": container with ID starting with 0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b not found: ID does not exist" containerID="0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b"
Jul 29 18:02:30.825079 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.825022    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b"} err="failed to get container status \"0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b\": rpc error: code = NotFound desc = could not find container \"0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b\": container with ID starting with 0395f6eab93b1ca5dd11071fa25f9d1026aa8984a254871f5fbb38a501ccc83b not found: ID does not exist"
Jul 29 18:02:30.835432 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.835409    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"]
Jul 29 18:02:30.838680 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:30.838658    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wxlbx"]
Jul 29 18:02:32.282025 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:32.281994    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fb53c38-c4ec-4afb-85c9-c28dda3b2360" path="/var/lib/kubelet/pods/7fb53c38-c4ec-4afb-85c9-c28dda3b2360/volumes"
Jul 29 18:02:34.644358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.644325    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"]
Jul 29 18:02:34.644746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.644714    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fb53c38-c4ec-4afb-85c9-c28dda3b2360" containerName="limitador"
Jul 29 18:02:34.644746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.644726    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fb53c38-c4ec-4afb-85c9-c28dda3b2360" containerName="limitador"
Jul 29 18:02:34.644835 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.644784    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fb53c38-c4ec-4afb-85c9-c28dda3b2360" containerName="limitador"
Jul 29 18:02:34.648522 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.648494    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:34.652662 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.652639    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 18:02:34.652789 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.652724    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 18:02:34.652895 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.652877    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 18:02:34.653035 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.653023    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-m8d2m\""
Jul 29 18:02:34.664083 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.664065    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"]
Jul 29 18:02:34.729592 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.729560    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmq4f\" (UniqueName: \"kubernetes.io/projected/c05ad0b5-1385-4f7a-9572-c69d92abcd55-kube-api-access-wmq4f\") pod \"postgres-69fb4594cf-mtrrc\" (UID: \"c05ad0b5-1385-4f7a-9572-c69d92abcd55\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:34.729805 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.729603    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c05ad0b5-1385-4f7a-9572-c69d92abcd55-data\") pod \"postgres-69fb4594cf-mtrrc\" (UID: \"c05ad0b5-1385-4f7a-9572-c69d92abcd55\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:34.830510 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.830472    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c05ad0b5-1385-4f7a-9572-c69d92abcd55-data\") pod \"postgres-69fb4594cf-mtrrc\" (UID: \"c05ad0b5-1385-4f7a-9572-c69d92abcd55\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:34.830679 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.830578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmq4f\" (UniqueName: \"kubernetes.io/projected/c05ad0b5-1385-4f7a-9572-c69d92abcd55-kube-api-access-wmq4f\") pod \"postgres-69fb4594cf-mtrrc\" (UID: \"c05ad0b5-1385-4f7a-9572-c69d92abcd55\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:34.830876 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.830857    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c05ad0b5-1385-4f7a-9572-c69d92abcd55-data\") pod \"postgres-69fb4594cf-mtrrc\" (UID: \"c05ad0b5-1385-4f7a-9572-c69d92abcd55\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:34.846249 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.846215    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmq4f\" (UniqueName: \"kubernetes.io/projected/c05ad0b5-1385-4f7a-9572-c69d92abcd55-kube-api-access-wmq4f\") pod \"postgres-69fb4594cf-mtrrc\" (UID: \"c05ad0b5-1385-4f7a-9572-c69d92abcd55\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:34.960327 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:34.960202    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:35.089078 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:35.089023    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"]
Jul 29 18:02:35.091326 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:02:35.091302    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc05ad0b5_1385_4f7a_9572_c69d92abcd55.slice/crio-3e1aad059d9961836194c9c22e5b83299246eb950719dcc847a97347ce4886bd WatchSource:0}: Error finding container 3e1aad059d9961836194c9c22e5b83299246eb950719dcc847a97347ce4886bd: Status 404 returned error can't find the container with id 3e1aad059d9961836194c9c22e5b83299246eb950719dcc847a97347ce4886bd
Jul 29 18:02:35.848647 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:35.848561    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc" event={"ID":"c05ad0b5-1385-4f7a-9572-c69d92abcd55","Type":"ContainerStarted","Data":"3e1aad059d9961836194c9c22e5b83299246eb950719dcc847a97347ce4886bd"}
Jul 29 18:02:40.875502 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:40.875465    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc" event={"ID":"c05ad0b5-1385-4f7a-9572-c69d92abcd55","Type":"ContainerStarted","Data":"677d6f01a0090b7109f17ee446f7ef389caac0eb0408fb0b21a71d0635e04251"}
Jul 29 18:02:40.875888 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:40.875622    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:40.892717 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:40.892665    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc" podStartSLOduration=2.159857899 podStartE2EDuration="6.892649744s" totalImagesPullingTime="4.732791845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:40.891844993 +0000 UTC m=+551.133806054" watchObservedRunningTime="2026-07-29 18:02:40.892649744 +0000 UTC m=+551.134610781"
Jul 29 18:02:46.909141 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:46.909111    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mtrrc"
Jul 29 18:02:47.448076 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.448042    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tm22j"]
Jul 29 18:02:47.470221 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.470184    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tm22j"]
Jul 29 18:02:47.470375 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.470305    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tm22j"
Jul 29 18:02:47.544950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.544917    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pcwl\" (UniqueName: \"kubernetes.io/projected/8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836-kube-api-access-4pcwl\") pod \"authorino-5d4b67c446-tm22j\" (UID: \"8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836\") " pod="kuadrant-system/authorino-5d4b67c446-tm22j"
Jul 29 18:02:47.634032 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.633998    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tm22j"]
Jul 29 18:02:47.634327 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:02:47.634304    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4pcwl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-tm22j" podUID="8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836"
Jul 29 18:02:47.645440 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.645415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pcwl\" (UniqueName: \"kubernetes.io/projected/8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836-kube-api-access-4pcwl\") pod \"authorino-5d4b67c446-tm22j\" (UID: \"8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836\") " pod="kuadrant-system/authorino-5d4b67c446-tm22j"
Jul 29 18:02:47.659025 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.658993    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pcwl\" (UniqueName: \"kubernetes.io/projected/8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836-kube-api-access-4pcwl\") pod \"authorino-5d4b67c446-tm22j\" (UID: \"8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836\") " pod="kuadrant-system/authorino-5d4b67c446-tm22j"
Jul 29 18:02:47.665936 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.665910    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-2mv86"]
Jul 29 18:02:47.675862 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.675838    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.677616 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.677594    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-2mv86"]
Jul 29 18:02:47.677759 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.677738    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 18:02:47.699751 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.699686    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-2mv86"]
Jul 29 18:02:47.699969 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:02:47.699950    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gz6qw tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-gz6qw tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-2mv86" podUID="2795a921-6581-4bd5-8e60-bd61c63e3a8a"
Jul 29 18:02:47.725218 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.725190    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-589c8b6bb5-xcpds"]
Jul 29 18:02:47.746224 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.746186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2795a921-6581-4bd5-8e60-bd61c63e3a8a-tls-cert\") pod \"authorino-76877994-2mv86\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") " pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.746407 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.746306    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz6qw\" (UniqueName: \"kubernetes.io/projected/2795a921-6581-4bd5-8e60-bd61c63e3a8a-kube-api-access-gz6qw\") pod \"authorino-76877994-2mv86\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") " pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.746890 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.746872    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-589c8b6bb5-xcpds"]
Jul 29 18:02:47.746969 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.746959    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:47.846854 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.846818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2795a921-6581-4bd5-8e60-bd61c63e3a8a-tls-cert\") pod \"authorino-76877994-2mv86\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") " pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.847059 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.846888    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d14d3176-8626-4dfe-aefd-2d1062abaf09-tls-cert\") pod \"authorino-589c8b6bb5-xcpds\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") " pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:47.847059 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.846945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz6qw\" (UniqueName: \"kubernetes.io/projected/2795a921-6581-4bd5-8e60-bd61c63e3a8a-kube-api-access-gz6qw\") pod \"authorino-76877994-2mv86\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") " pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.847059 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.847025    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpbfs\" (UniqueName: \"kubernetes.io/projected/d14d3176-8626-4dfe-aefd-2d1062abaf09-kube-api-access-wpbfs\") pod \"authorino-589c8b6bb5-xcpds\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") " pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:47.849513 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.849489    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2795a921-6581-4bd5-8e60-bd61c63e3a8a-tls-cert\") pod \"authorino-76877994-2mv86\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") " pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.855236 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.855213    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz6qw\" (UniqueName: \"kubernetes.io/projected/2795a921-6581-4bd5-8e60-bd61c63e3a8a-kube-api-access-gz6qw\") pod \"authorino-76877994-2mv86\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") " pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.907887 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.907850    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tm22j"
Jul 29 18:02:47.908072 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.907850    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.928410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.928379    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tm22j"
Jul 29 18:02:47.942278 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.942239    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:47.948003 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.947984    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pcwl\" (UniqueName: \"kubernetes.io/projected/8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836-kube-api-access-4pcwl\") pod \"8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836\" (UID: \"8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836\") "
Jul 29 18:02:47.948151 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.948135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbfs\" (UniqueName: \"kubernetes.io/projected/d14d3176-8626-4dfe-aefd-2d1062abaf09-kube-api-access-wpbfs\") pod \"authorino-589c8b6bb5-xcpds\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") " pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:47.948218 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.948186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d14d3176-8626-4dfe-aefd-2d1062abaf09-tls-cert\") pod \"authorino-589c8b6bb5-xcpds\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") " pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:47.950217 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.950157    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836-kube-api-access-4pcwl" (OuterVolumeSpecName: "kube-api-access-4pcwl") pod "8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836" (UID: "8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836"). InnerVolumeSpecName "kube-api-access-4pcwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:02:47.950588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.950572    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d14d3176-8626-4dfe-aefd-2d1062abaf09-tls-cert\") pod \"authorino-589c8b6bb5-xcpds\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") " pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:47.955245 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:47.955228    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbfs\" (UniqueName: \"kubernetes.io/projected/d14d3176-8626-4dfe-aefd-2d1062abaf09-kube-api-access-wpbfs\") pod \"authorino-589c8b6bb5-xcpds\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") " pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:48.049329 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.049294    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2795a921-6581-4bd5-8e60-bd61c63e3a8a-tls-cert\") pod \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") "
Jul 29 18:02:48.049492 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.049354    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz6qw\" (UniqueName: \"kubernetes.io/projected/2795a921-6581-4bd5-8e60-bd61c63e3a8a-kube-api-access-gz6qw\") pod \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\" (UID: \"2795a921-6581-4bd5-8e60-bd61c63e3a8a\") "
Jul 29 18:02:48.049561 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.049549    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pcwl\" (UniqueName: \"kubernetes.io/projected/8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836-kube-api-access-4pcwl\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:48.051456 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.051429    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2795a921-6581-4bd5-8e60-bd61c63e3a8a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2795a921-6581-4bd5-8e60-bd61c63e3a8a" (UID: "2795a921-6581-4bd5-8e60-bd61c63e3a8a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:02:48.051563 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.051525    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2795a921-6581-4bd5-8e60-bd61c63e3a8a-kube-api-access-gz6qw" (OuterVolumeSpecName: "kube-api-access-gz6qw") pod "2795a921-6581-4bd5-8e60-bd61c63e3a8a" (UID: "2795a921-6581-4bd5-8e60-bd61c63e3a8a"). InnerVolumeSpecName "kube-api-access-gz6qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:02:48.055624 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.055606    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:02:48.150027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.150000    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2795a921-6581-4bd5-8e60-bd61c63e3a8a-tls-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:48.150102 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.150036    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz6qw\" (UniqueName: \"kubernetes.io/projected/2795a921-6581-4bd5-8e60-bd61c63e3a8a-kube-api-access-gz6qw\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:48.196505 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.196480    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-589c8b6bb5-xcpds"]
Jul 29 18:02:48.201561 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:02:48.201484    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd14d3176_8626_4dfe_aefd_2d1062abaf09.slice/crio-7627712cd25fe259003f050c33db589e01630474aacca04d5f6d4ae59071ecbd WatchSource:0}: Error finding container 7627712cd25fe259003f050c33db589e01630474aacca04d5f6d4ae59071ecbd: Status 404 returned error can't find the container with id 7627712cd25fe259003f050c33db589e01630474aacca04d5f6d4ae59071ecbd
Jul 29 18:02:48.913410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.913386    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-2mv86"
Jul 29 18:02:48.913410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.913400    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-589c8b6bb5-xcpds" event={"ID":"d14d3176-8626-4dfe-aefd-2d1062abaf09","Type":"ContainerStarted","Data":"7627712cd25fe259003f050c33db589e01630474aacca04d5f6d4ae59071ecbd"}
Jul 29 18:02:48.913594 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.913569    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tm22j"
Jul 29 18:02:48.938457 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.938435    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-2mv86"]
Jul 29 18:02:48.940055 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.940034    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-2mv86"]
Jul 29 18:02:48.959331 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.959294    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tm22j"]
Jul 29 18:02:48.964596 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:48.964565    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tm22j"]
Jul 29 18:02:49.919162 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:49.919130    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-589c8b6bb5-xcpds" event={"ID":"d14d3176-8626-4dfe-aefd-2d1062abaf09","Type":"ContainerStarted","Data":"a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467"}
Jul 29 18:02:49.933707 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:49.933653    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-589c8b6bb5-xcpds" podStartSLOduration=2.297644897 podStartE2EDuration="2.933635309s" totalImagesPullingTime="635.990412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:49.931760493 +0000 UTC m=+560.173721528" watchObservedRunningTime="2026-07-29 18:02:49.933635309 +0000 UTC m=+560.175596380"
Jul 29 18:02:49.958853 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:49.958818    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wxfdv"]
Jul 29 18:02:49.959285 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:49.959062    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wxfdv" podUID="b10daf60-4de2-41fd-a962-ef31c7ddc689" containerName="authorino" containerID="cri-o://f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632" gracePeriod=30
Jul 29 18:02:50.281462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.281430    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2795a921-6581-4bd5-8e60-bd61c63e3a8a" path="/var/lib/kubelet/pods/2795a921-6581-4bd5-8e60-bd61c63e3a8a/volumes"
Jul 29 18:02:50.281786 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.281768    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836" path="/var/lib/kubelet/pods/8eb0ce96-0dbf-4cdf-bd9a-f484b5c90836/volumes"
Jul 29 18:02:50.295953 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.295923    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wxfdv"
Jul 29 18:02:50.369684 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.369644    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp9kd\" (UniqueName: \"kubernetes.io/projected/b10daf60-4de2-41fd-a962-ef31c7ddc689-kube-api-access-fp9kd\") pod \"b10daf60-4de2-41fd-a962-ef31c7ddc689\" (UID: \"b10daf60-4de2-41fd-a962-ef31c7ddc689\") "
Jul 29 18:02:50.371762 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.371732    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10daf60-4de2-41fd-a962-ef31c7ddc689-kube-api-access-fp9kd" (OuterVolumeSpecName: "kube-api-access-fp9kd") pod "b10daf60-4de2-41fd-a962-ef31c7ddc689" (UID: "b10daf60-4de2-41fd-a962-ef31c7ddc689"). InnerVolumeSpecName "kube-api-access-fp9kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:02:50.471194 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.471159    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp9kd\" (UniqueName: \"kubernetes.io/projected/b10daf60-4de2-41fd-a962-ef31c7ddc689-kube-api-access-fp9kd\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:02:50.924488 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.924450    2560 generic.go:356] "Generic (PLEG): container finished" podID="b10daf60-4de2-41fd-a962-ef31c7ddc689" containerID="f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632" exitCode=0
Jul 29 18:02:50.924687 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.924514    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wxfdv"
Jul 29 18:02:50.924687 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.924544    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wxfdv" event={"ID":"b10daf60-4de2-41fd-a962-ef31c7ddc689","Type":"ContainerDied","Data":"f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632"}
Jul 29 18:02:50.924687 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.924612    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wxfdv" event={"ID":"b10daf60-4de2-41fd-a962-ef31c7ddc689","Type":"ContainerDied","Data":"270c140f56fcc12d25faec405c47c5892ee3241a514d6ccfd282ebe2a3a26e86"}
Jul 29 18:02:50.924687 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.924641    2560 scope.go:117] "RemoveContainer" containerID="f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632"
Jul 29 18:02:50.933434 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.933414    2560 scope.go:117] "RemoveContainer" containerID="f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632"
Jul 29 18:02:50.933709 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:02:50.933689    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632\": container with ID starting with f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632 not found: ID does not exist" containerID="f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632"
Jul 29 18:02:50.933758 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.933717    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632"} err="failed to get container status \"f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632\": rpc error: code = NotFound desc = could not find container \"f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632\": container with ID starting with f0c417442c2e89ae49925db63c49137eb3cc04e5596f90e6248233ed9366d632 not found: ID does not exist"
Jul 29 18:02:50.943198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.943170    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wxfdv"]
Jul 29 18:02:50.949604 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:50.949581    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wxfdv"]
Jul 29 18:02:52.281913 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:52.281879    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b10daf60-4de2-41fd-a962-ef31c7ddc689" path="/var/lib/kubelet/pods/b10daf60-4de2-41fd-a962-ef31c7ddc689/volumes"
Jul 29 18:02:54.642167 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.642131    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-fpcg9"]
Jul 29 18:02:54.642567 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.642516    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b10daf60-4de2-41fd-a962-ef31c7ddc689" containerName="authorino"
Jul 29 18:02:54.642567 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.642530    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b10daf60-4de2-41fd-a962-ef31c7ddc689" containerName="authorino"
Jul 29 18:02:54.642637 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.642601    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b10daf60-4de2-41fd-a962-ef31c7ddc689" containerName="authorino"
Jul 29 18:02:54.647036 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.647019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:54.648915 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.648896    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7x8jm\""
Jul 29 18:02:54.649037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.648917    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 18:02:54.649037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.648928    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 18:02:54.655626 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.655602    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-fpcg9"]
Jul 29 18:02:54.710750 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.710717    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94a10467-6fa0-4bf6-9e41-f37cc2656e59-webhook-cert\") pod \"maas-controller-7dc687fcc6-fpcg9\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") " pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:54.710925 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.710772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjg9n\" (UniqueName: \"kubernetes.io/projected/94a10467-6fa0-4bf6-9e41-f37cc2656e59-kube-api-access-zjg9n\") pod \"maas-controller-7dc687fcc6-fpcg9\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") " pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:54.811962 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.811925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94a10467-6fa0-4bf6-9e41-f37cc2656e59-webhook-cert\") pod \"maas-controller-7dc687fcc6-fpcg9\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") " pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:54.812162 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.811982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjg9n\" (UniqueName: \"kubernetes.io/projected/94a10467-6fa0-4bf6-9e41-f37cc2656e59-kube-api-access-zjg9n\") pod \"maas-controller-7dc687fcc6-fpcg9\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") " pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:54.814692 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.814659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94a10467-6fa0-4bf6-9e41-f37cc2656e59-webhook-cert\") pod \"maas-controller-7dc687fcc6-fpcg9\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") " pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:54.819597 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.819577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjg9n\" (UniqueName: \"kubernetes.io/projected/94a10467-6fa0-4bf6-9e41-f37cc2656e59-kube-api-access-zjg9n\") pod \"maas-controller-7dc687fcc6-fpcg9\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") " pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:54.958110 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:54.958025    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:55.100717 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:55.100694    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-fpcg9"]
Jul 29 18:02:55.102922 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:02:55.102895    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a10467_6fa0_4bf6_9e41_f37cc2656e59.slice/crio-04a79b31b6c0684b7d47e9f9f861343fb2f639390982951abd0003b687cffc96 WatchSource:0}: Error finding container 04a79b31b6c0684b7d47e9f9f861343fb2f639390982951abd0003b687cffc96: Status 404 returned error can't find the container with id 04a79b31b6c0684b7d47e9f9f861343fb2f639390982951abd0003b687cffc96
Jul 29 18:02:55.951667 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:55.951628    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9" event={"ID":"94a10467-6fa0-4bf6-9e41-f37cc2656e59","Type":"ContainerStarted","Data":"04a79b31b6c0684b7d47e9f9f861343fb2f639390982951abd0003b687cffc96"}
Jul 29 18:02:57.963861 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:57.963779    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9" event={"ID":"94a10467-6fa0-4bf6-9e41-f37cc2656e59","Type":"ContainerStarted","Data":"082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68"}
Jul 29 18:02:57.964232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:57.963929    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:02:57.980140 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:02:57.980083    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9" podStartSLOduration=1.582815641 podStartE2EDuration="3.980065139s" totalImagesPullingTime="2.397249498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:57.979003023 +0000 UTC m=+568.220964061" watchObservedRunningTime="2026-07-29 18:02:57.980065139 +0000 UTC m=+568.222026177"
Jul 29 18:03:08.971009 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:08.970928    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:03:30.045534 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.045503    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"]
Jul 29 18:03:30.052251 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.052227    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.054424 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.054404    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 18:03:30.054656 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.054638    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nt97v\""
Jul 29 18:03:30.058576 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.058552    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"]
Jul 29 18:03:30.145227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.145201    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55zrb\" (UniqueName: \"kubernetes.io/projected/7d38d4da-a4d2-4205-aadf-6b49268ef493-kube-api-access-55zrb\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.145424 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.145255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d38d4da-a4d2-4205-aadf-6b49268ef493-openshift-service-ca\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.145424 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.145357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d38d4da-a4d2-4205-aadf-6b49268ef493-maas-api-tls\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.150974 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.150949    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:03:30.152720 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.152695    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:03:30.246182 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.246145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d38d4da-a4d2-4205-aadf-6b49268ef493-openshift-service-ca\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.246392 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.246248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d38d4da-a4d2-4205-aadf-6b49268ef493-maas-api-tls\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.246392 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.246364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55zrb\" (UniqueName: \"kubernetes.io/projected/7d38d4da-a4d2-4205-aadf-6b49268ef493-kube-api-access-55zrb\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.246834 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.246812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d38d4da-a4d2-4205-aadf-6b49268ef493-openshift-service-ca\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.248137 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.248123    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 18:03:30.253996 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.253970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55zrb\" (UniqueName: \"kubernetes.io/projected/7d38d4da-a4d2-4205-aadf-6b49268ef493-kube-api-access-55zrb\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.258844 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.258823    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d38d4da-a4d2-4205-aadf-6b49268ef493-maas-api-tls\") pod \"maas-api-7b5699ddd4-qwp6v\" (UID: \"7d38d4da-a4d2-4205-aadf-6b49268ef493\") " pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.365891 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.365809    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nt97v\""
Jul 29 18:03:30.374297 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.374252    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:30.524677 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:30.524655    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"]
Jul 29 18:03:30.526880 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:03:30.526840    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d38d4da_a4d2_4205_aadf_6b49268ef493.slice/crio-150d894dc329d71f9a6e19fee125691a0b60efda02578a1494e37d1de3f8411a WatchSource:0}: Error finding container 150d894dc329d71f9a6e19fee125691a0b60efda02578a1494e37d1de3f8411a: Status 404 returned error can't find the container with id 150d894dc329d71f9a6e19fee125691a0b60efda02578a1494e37d1de3f8411a
Jul 29 18:03:31.121163 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:31.121130    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v" event={"ID":"7d38d4da-a4d2-4205-aadf-6b49268ef493","Type":"ContainerStarted","Data":"150d894dc329d71f9a6e19fee125691a0b60efda02578a1494e37d1de3f8411a"}
Jul 29 18:03:33.134326 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:33.134260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v" event={"ID":"7d38d4da-a4d2-4205-aadf-6b49268ef493","Type":"ContainerStarted","Data":"cc4e1e4fd6d5c35ac22c3552948ed8ce77c0d075684ece2fcf3fa1faacb358e3"}
Jul 29 18:03:33.134766 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:33.134399    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:03:33.149940 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:33.149893    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v" podStartSLOduration=1.281537536 podStartE2EDuration="3.149877985s" totalImagesPullingTime="1.868340449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:03:33.14818559 +0000 UTC m=+603.390146628" watchObservedRunningTime="2026-07-29 18:03:33.149877985 +0000 UTC m=+603.391839024"
Jul 29 18:03:39.144161 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:03:39.144129    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7b5699ddd4-qwp6v"
Jul 29 18:04:10.615332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.615296    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"]
Jul 29 18:04:10.618124 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.618104    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.620286 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.620245    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 18:04:10.620695 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.620677    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 18:04:10.620781 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.620707    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-68jvk\""
Jul 29 18:04:10.620781 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.620707    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 18:04:10.629237 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.629212    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"]
Jul 29 18:04:10.721557 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.721521    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.721736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.721561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.721736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.721590    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e7c809-dc66-4618-843a-ca196d8c1b1f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.721736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.721658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.721736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.721703    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.721736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.721719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq8dd\" (UniqueName: \"kubernetes.io/projected/a0e7c809-dc66-4618-843a-ca196d8c1b1f-kube-api-access-wq8dd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.822834 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.822801    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.822845    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.822868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq8dd\" (UniqueName: \"kubernetes.io/projected/a0e7c809-dc66-4618-843a-ca196d8c1b1f-kube-api-access-wq8dd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.822925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.822956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823233 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.823084    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e7c809-dc66-4618-843a-ca196d8c1b1f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823359 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.823338    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823429 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.823356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.823486 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.823443    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.825205 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.825186    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e7c809-dc66-4618-843a-ca196d8c1b1f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.825614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.825593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e7c809-dc66-4618-843a-ca196d8c1b1f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.829943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.829922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq8dd\" (UniqueName: \"kubernetes.io/projected/a0e7c809-dc66-4618-843a-ca196d8c1b1f-kube-api-access-wq8dd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mclp5\" (UID: \"a0e7c809-dc66-4618-843a-ca196d8c1b1f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:10.929841 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:10.929756    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:11.067697 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:11.067669    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"]
Jul 29 18:04:11.070186 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:04:11.070159    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e7c809_dc66_4618_843a_ca196d8c1b1f.slice/crio-29a683f08f705857c3940bfabc4b6534c304a9cb6762e995d439b78a05da81e0 WatchSource:0}: Error finding container 29a683f08f705857c3940bfabc4b6534c304a9cb6762e995d439b78a05da81e0: Status 404 returned error can't find the container with id 29a683f08f705857c3940bfabc4b6534c304a9cb6762e995d439b78a05da81e0
Jul 29 18:04:11.071546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:11.071525    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:04:11.317357 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:11.317320    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5" event={"ID":"a0e7c809-dc66-4618-843a-ca196d8c1b1f","Type":"ContainerStarted","Data":"29a683f08f705857c3940bfabc4b6534c304a9cb6762e995d439b78a05da81e0"}
Jul 29 18:04:13.328997 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:13.328964    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5" event={"ID":"a0e7c809-dc66-4618-843a-ca196d8c1b1f","Type":"ContainerStarted","Data":"3274912b8b3d600d7981f83e915f46ebebd422d5acdf6b5a1dc4bd4050de4622"}
Jul 29 18:04:13.346114 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:13.346066    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5" podStartSLOduration=2.11998144 podStartE2EDuration="3.346049608s" totalImagesPullingTime="1.226068168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:04:13.343752332 +0000 UTC m=+643.585713370" watchObservedRunningTime="2026-07-29 18:04:13.346049608 +0000 UTC m=+643.588010645"
Jul 29 18:04:20.930489 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:20.930447    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:20.930938 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:20.930782    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:20.943061 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:20.943033    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:21.376213 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:21.376188    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mclp5"
Jul 29 18:04:22.117655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.117621    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"]
Jul 29 18:04:22.121132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.121112    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.122906 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.122878    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 29 18:04:22.130311 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.130289    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"]
Jul 29 18:04:22.229833 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.229796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.230009 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.229861    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.230009 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.229944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.230009 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.229990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.230118 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.230025    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47c341ee-d4be-4ec7-9d64-0c047bed8da7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.230118 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.230046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n62sf\" (UniqueName: \"kubernetes.io/projected/47c341ee-d4be-4ec7-9d64-0c047bed8da7-kube-api-access-n62sf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331236 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331195    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331456 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331456 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331313    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47c341ee-d4be-4ec7-9d64-0c047bed8da7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331456 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n62sf\" (UniqueName: \"kubernetes.io/projected/47c341ee-d4be-4ec7-9d64-0c047bed8da7-kube-api-access-n62sf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331456 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331369    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331456 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331402    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331760 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331737    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331827 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331805    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.331875 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.331860    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.333609 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.333587    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47c341ee-d4be-4ec7-9d64-0c047bed8da7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.333862 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.333844    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47c341ee-d4be-4ec7-9d64-0c047bed8da7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.344468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.344448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n62sf\" (UniqueName: \"kubernetes.io/projected/47c341ee-d4be-4ec7-9d64-0c047bed8da7-kube-api-access-n62sf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2\" (UID: \"47c341ee-d4be-4ec7-9d64-0c047bed8da7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.432107 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.432021    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:22.565310 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:22.565288    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"]
Jul 29 18:04:22.567919 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:04:22.567883    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c341ee_d4be_4ec7_9d64_0c047bed8da7.slice/crio-943cb2a9f0bd5d88a1aa8bb8850e87f4a833d902f6aa0788e1f9b5a33d8999d3 WatchSource:0}: Error finding container 943cb2a9f0bd5d88a1aa8bb8850e87f4a833d902f6aa0788e1f9b5a33d8999d3: Status 404 returned error can't find the container with id 943cb2a9f0bd5d88a1aa8bb8850e87f4a833d902f6aa0788e1f9b5a33d8999d3
Jul 29 18:04:23.375233 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:23.375195    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2" event={"ID":"47c341ee-d4be-4ec7-9d64-0c047bed8da7","Type":"ContainerStarted","Data":"211f8bc4c888632fbbe59528cf241adb1700b87e9f863bfc53e136c2e3dcc4b6"}
Jul 29 18:04:23.375652 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:23.375256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2" event={"ID":"47c341ee-d4be-4ec7-9d64-0c047bed8da7","Type":"ContainerStarted","Data":"943cb2a9f0bd5d88a1aa8bb8850e87f4a833d902f6aa0788e1f9b5a33d8999d3"}
Jul 29 18:04:23.391866 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:23.391815    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2" podStartSLOduration=1.108225714 podStartE2EDuration="1.39179892s" totalImagesPullingTime="283.573206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:04:23.391116524 +0000 UTC m=+653.633077564" watchObservedRunningTime="2026-07-29 18:04:23.39179892 +0000 UTC m=+653.633759957"
Jul 29 18:04:32.432837 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:32.432798    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:32.433431 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:32.432970    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:32.445467 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:32.445443    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:04:33.435867 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:04:33.435829    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2"
Jul 29 18:05:54.735998 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:54.735965    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-589c8b6bb5-xcpds"]
Jul 29 18:05:54.736608 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:54.736200    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-589c8b6bb5-xcpds" podUID="d14d3176-8626-4dfe-aefd-2d1062abaf09" containerName="authorino" containerID="cri-o://a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467" gracePeriod=30
Jul 29 18:05:54.992562 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:54.992513    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:05:55.123590 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.123552    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpbfs\" (UniqueName: \"kubernetes.io/projected/d14d3176-8626-4dfe-aefd-2d1062abaf09-kube-api-access-wpbfs\") pod \"d14d3176-8626-4dfe-aefd-2d1062abaf09\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") "
Jul 29 18:05:55.123782 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.123613    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d14d3176-8626-4dfe-aefd-2d1062abaf09-tls-cert\") pod \"d14d3176-8626-4dfe-aefd-2d1062abaf09\" (UID: \"d14d3176-8626-4dfe-aefd-2d1062abaf09\") "
Jul 29 18:05:55.125854 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.125829    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d14d3176-8626-4dfe-aefd-2d1062abaf09-kube-api-access-wpbfs" (OuterVolumeSpecName: "kube-api-access-wpbfs") pod "d14d3176-8626-4dfe-aefd-2d1062abaf09" (UID: "d14d3176-8626-4dfe-aefd-2d1062abaf09"). InnerVolumeSpecName "kube-api-access-wpbfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:05:55.134532 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.134505    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d14d3176-8626-4dfe-aefd-2d1062abaf09-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d14d3176-8626-4dfe-aefd-2d1062abaf09" (UID: "d14d3176-8626-4dfe-aefd-2d1062abaf09"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:05:55.224577 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.224533    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpbfs\" (UniqueName: \"kubernetes.io/projected/d14d3176-8626-4dfe-aefd-2d1062abaf09-kube-api-access-wpbfs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:05:55.224577 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.224570    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d14d3176-8626-4dfe-aefd-2d1062abaf09-tls-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:05:55.820833 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.820799    2560 generic.go:356] "Generic (PLEG): container finished" podID="d14d3176-8626-4dfe-aefd-2d1062abaf09" containerID="a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467" exitCode=0
Jul 29 18:05:55.821325 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.820849    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-589c8b6bb5-xcpds"
Jul 29 18:05:55.821325 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.820869    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-589c8b6bb5-xcpds" event={"ID":"d14d3176-8626-4dfe-aefd-2d1062abaf09","Type":"ContainerDied","Data":"a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467"}
Jul 29 18:05:55.821325 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.820899    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-589c8b6bb5-xcpds" event={"ID":"d14d3176-8626-4dfe-aefd-2d1062abaf09","Type":"ContainerDied","Data":"7627712cd25fe259003f050c33db589e01630474aacca04d5f6d4ae59071ecbd"}
Jul 29 18:05:55.821325 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.820915    2560 scope.go:117] "RemoveContainer" containerID="a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467"
Jul 29 18:05:55.836973 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.836949    2560 scope.go:117] "RemoveContainer" containerID="a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467"
Jul 29 18:05:55.837279 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:05:55.837247    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467\": container with ID starting with a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467 not found: ID does not exist" containerID="a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467"
Jul 29 18:05:55.837377 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.837329    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467"} err="failed to get container status \"a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467\": rpc error: code = NotFound desc = could not find container \"a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467\": container with ID starting with a9f958f8a7add9c7cca510292af0a8c97a3ed1a741999c93f5967e334ad11467 not found: ID does not exist"
Jul 29 18:05:55.846173 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.846145    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-589c8b6bb5-xcpds"]
Jul 29 18:05:55.852175 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:55.852154    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-589c8b6bb5-xcpds"]
Jul 29 18:05:56.282187 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:05:56.282152    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d14d3176-8626-4dfe-aefd-2d1062abaf09" path="/var/lib/kubelet/pods/d14d3176-8626-4dfe-aefd-2d1062abaf09/volumes"
Jul 29 18:06:40.469758 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.469722    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-fpcg9"]
Jul 29 18:06:40.470224 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.470006    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9" podUID="94a10467-6fa0-4bf6-9e41-f37cc2656e59" containerName="manager" containerID="cri-o://082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68" gracePeriod=30
Jul 29 18:06:40.721237 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.721175    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:06:40.772252 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.772221    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94a10467-6fa0-4bf6-9e41-f37cc2656e59-webhook-cert\") pod \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") "
Jul 29 18:06:40.772449 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.772317    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjg9n\" (UniqueName: \"kubernetes.io/projected/94a10467-6fa0-4bf6-9e41-f37cc2656e59-kube-api-access-zjg9n\") pod \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\" (UID: \"94a10467-6fa0-4bf6-9e41-f37cc2656e59\") "
Jul 29 18:06:40.774564 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.774528    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94a10467-6fa0-4bf6-9e41-f37cc2656e59-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "94a10467-6fa0-4bf6-9e41-f37cc2656e59" (UID: "94a10467-6fa0-4bf6-9e41-f37cc2656e59"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:06:40.774670 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.774592    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a10467-6fa0-4bf6-9e41-f37cc2656e59-kube-api-access-zjg9n" (OuterVolumeSpecName: "kube-api-access-zjg9n") pod "94a10467-6fa0-4bf6-9e41-f37cc2656e59" (UID: "94a10467-6fa0-4bf6-9e41-f37cc2656e59"). InnerVolumeSpecName "kube-api-access-zjg9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:06:40.873378 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.873339    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94a10467-6fa0-4bf6-9e41-f37cc2656e59-webhook-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:06:40.873378 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:40.873371    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjg9n\" (UniqueName: \"kubernetes.io/projected/94a10467-6fa0-4bf6-9e41-f37cc2656e59-kube-api-access-zjg9n\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:06:41.041940 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.041907    2560 generic.go:356] "Generic (PLEG): container finished" podID="94a10467-6fa0-4bf6-9e41-f37cc2656e59" containerID="082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68" exitCode=0
Jul 29 18:06:41.042154 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.041960    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9"
Jul 29 18:06:41.042154 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.042003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9" event={"ID":"94a10467-6fa0-4bf6-9e41-f37cc2656e59","Type":"ContainerDied","Data":"082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68"}
Jul 29 18:06:41.042154 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.042062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-fpcg9" event={"ID":"94a10467-6fa0-4bf6-9e41-f37cc2656e59","Type":"ContainerDied","Data":"04a79b31b6c0684b7d47e9f9f861343fb2f639390982951abd0003b687cffc96"}
Jul 29 18:06:41.042154 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.042090    2560 scope.go:117] "RemoveContainer" containerID="082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68"
Jul 29 18:06:41.052048 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.052028    2560 scope.go:117] "RemoveContainer" containerID="082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68"
Jul 29 18:06:41.052367 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:06:41.052348    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68\": container with ID starting with 082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68 not found: ID does not exist" containerID="082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68"
Jul 29 18:06:41.052424 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.052376    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68"} err="failed to get container status \"082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68\": rpc error: code = NotFound desc = could not find container \"082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68\": container with ID starting with 082c993791ac1809c12e05a6536e494929ccbfde7a140d1bb8804afa1a230e68 not found: ID does not exist"
Jul 29 18:06:41.060765 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.060743    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-fpcg9"]
Jul 29 18:06:41.063938 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:41.063911    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-fpcg9"]
Jul 29 18:06:42.281716 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:06:42.281682    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94a10467-6fa0-4bf6-9e41-f37cc2656e59" path="/var/lib/kubelet/pods/94a10467-6fa0-4bf6-9e41-f37cc2656e59/volumes"
Jul 29 18:07:41.846903 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.846870    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-nlpfd"]
Jul 29 18:07:41.847418 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.847280    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d14d3176-8626-4dfe-aefd-2d1062abaf09" containerName="authorino"
Jul 29 18:07:41.847418 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.847292    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d14d3176-8626-4dfe-aefd-2d1062abaf09" containerName="authorino"
Jul 29 18:07:41.847418 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.847300    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94a10467-6fa0-4bf6-9e41-f37cc2656e59" containerName="manager"
Jul 29 18:07:41.847418 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.847305    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="94a10467-6fa0-4bf6-9e41-f37cc2656e59" containerName="manager"
Jul 29 18:07:41.847418 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.847369    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d14d3176-8626-4dfe-aefd-2d1062abaf09" containerName="authorino"
Jul 29 18:07:41.847418 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.847378    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="94a10467-6fa0-4bf6-9e41-f37cc2656e59" containerName="manager"
Jul 29 18:07:41.851598 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.851578    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:41.854178 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.854156    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 18:07:41.854308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.854205    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 18:07:41.854308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.854301    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7x8jm\""
Jul 29 18:07:41.862192 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.862169    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-nlpfd"]
Jul 29 18:07:41.957868 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.957834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-webhook-cert\") pod \"maas-controller-7dc687fcc6-nlpfd\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") " pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:41.958037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:41.957877    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8bhw\" (UniqueName: \"kubernetes.io/projected/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-kube-api-access-g8bhw\") pod \"maas-controller-7dc687fcc6-nlpfd\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") " pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:42.059316 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:42.059258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-webhook-cert\") pod \"maas-controller-7dc687fcc6-nlpfd\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") " pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:42.059468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:42.059326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8bhw\" (UniqueName: \"kubernetes.io/projected/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-kube-api-access-g8bhw\") pod \"maas-controller-7dc687fcc6-nlpfd\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") " pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:42.061853 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:42.061825    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-webhook-cert\") pod \"maas-controller-7dc687fcc6-nlpfd\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") " pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:42.066963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:42.066942    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8bhw\" (UniqueName: \"kubernetes.io/projected/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-kube-api-access-g8bhw\") pod \"maas-controller-7dc687fcc6-nlpfd\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") " pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:42.162924 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:42.162840    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:42.503224 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:42.503198    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-nlpfd"]
Jul 29 18:07:42.504586 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:07:42.504560    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a11e2dd_41b6_4ea4_94b2_2831457c04c1.slice/crio-18ed8fc410bb4514c078f870f7dad6ec7507d0a9997eb441501a7239519b7266 WatchSource:0}: Error finding container 18ed8fc410bb4514c078f870f7dad6ec7507d0a9997eb441501a7239519b7266: Status 404 returned error can't find the container with id 18ed8fc410bb4514c078f870f7dad6ec7507d0a9997eb441501a7239519b7266
Jul 29 18:07:43.324818 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:43.324785    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd" event={"ID":"1a11e2dd-41b6-4ea4-94b2-2831457c04c1","Type":"ContainerStarted","Data":"480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220"}
Jul 29 18:07:43.325227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:43.324863    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd" event={"ID":"1a11e2dd-41b6-4ea4-94b2-2831457c04c1","Type":"ContainerStarted","Data":"18ed8fc410bb4514c078f870f7dad6ec7507d0a9997eb441501a7239519b7266"}
Jul 29 18:07:43.325227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:43.324905    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:43.343286 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:43.343219    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd" podStartSLOduration=1.702669823 podStartE2EDuration="2.343202353s" totalImagesPullingTime="640.53253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:07:43.340984981 +0000 UTC m=+853.582946018" watchObservedRunningTime="2026-07-29 18:07:43.343202353 +0000 UTC m=+853.585163391"
Jul 29 18:07:54.331316 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:54.331260    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:58.944865 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:58.944831    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-nlpfd"]
Jul 29 18:07:58.945289 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:58.945022    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd" podUID="1a11e2dd-41b6-4ea4-94b2-2831457c04c1" containerName="manager" containerID="cri-o://480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220" gracePeriod=30
Jul 29 18:07:59.193653 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.193630    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:59.221874 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.221795    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8bhw\" (UniqueName: \"kubernetes.io/projected/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-kube-api-access-g8bhw\") pod \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") "
Jul 29 18:07:59.221874 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.221864    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-webhook-cert\") pod \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\" (UID: \"1a11e2dd-41b6-4ea4-94b2-2831457c04c1\") "
Jul 29 18:07:59.223897 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.223870    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1a11e2dd-41b6-4ea4-94b2-2831457c04c1" (UID: "1a11e2dd-41b6-4ea4-94b2-2831457c04c1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:07:59.224009 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.223985    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-kube-api-access-g8bhw" (OuterVolumeSpecName: "kube-api-access-g8bhw") pod "1a11e2dd-41b6-4ea4-94b2-2831457c04c1" (UID: "1a11e2dd-41b6-4ea4-94b2-2831457c04c1"). InnerVolumeSpecName "kube-api-access-g8bhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:07:59.323334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.323303    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8bhw\" (UniqueName: \"kubernetes.io/projected/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-kube-api-access-g8bhw\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:07:59.323334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.323333    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a11e2dd-41b6-4ea4-94b2-2831457c04c1-webhook-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:07:59.397617 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.397582    2560 generic.go:356] "Generic (PLEG): container finished" podID="1a11e2dd-41b6-4ea4-94b2-2831457c04c1" containerID="480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220" exitCode=0
Jul 29 18:07:59.397785 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.397628    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd"
Jul 29 18:07:59.397785 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.397666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd" event={"ID":"1a11e2dd-41b6-4ea4-94b2-2831457c04c1","Type":"ContainerDied","Data":"480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220"}
Jul 29 18:07:59.397785 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.397715    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-nlpfd" event={"ID":"1a11e2dd-41b6-4ea4-94b2-2831457c04c1","Type":"ContainerDied","Data":"18ed8fc410bb4514c078f870f7dad6ec7507d0a9997eb441501a7239519b7266"}
Jul 29 18:07:59.397785 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.397742    2560 scope.go:117] "RemoveContainer" containerID="480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220"
Jul 29 18:07:59.406771 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.406753    2560 scope.go:117] "RemoveContainer" containerID="480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220"
Jul 29 18:07:59.407022 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:07:59.407004    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220\": container with ID starting with 480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220 not found: ID does not exist" containerID="480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220"
Jul 29 18:07:59.407068 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.407032    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220"} err="failed to get container status \"480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220\": rpc error: code = NotFound desc = could not find container \"480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220\": container with ID starting with 480adc8e015b7f16d875b475ded78a2fea083c548ecfa19e8fbe59f975fe0220 not found: ID does not exist"
Jul 29 18:07:59.417628 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.417592    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-nlpfd"]
Jul 29 18:07:59.421112 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:07:59.421084    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-nlpfd"]
Jul 29 18:08:00.180517 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.180485    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-5msks"]
Jul 29 18:08:00.180925 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.180910    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a11e2dd-41b6-4ea4-94b2-2831457c04c1" containerName="manager"
Jul 29 18:08:00.180968 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.180928    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a11e2dd-41b6-4ea4-94b2-2831457c04c1" containerName="manager"
Jul 29 18:08:00.181011 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.181002    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a11e2dd-41b6-4ea4-94b2-2831457c04c1" containerName="manager"
Jul 29 18:08:00.185526 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.185504    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.187822 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.187793    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 18:08:00.187953 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.187794    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 18:08:00.187953 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.187793    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7x8jm\""
Jul 29 18:08:00.193958 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.193931    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-5msks"]
Jul 29 18:08:00.231691 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.231653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh75d\" (UniqueName: \"kubernetes.io/projected/e19e8e60-6d2c-4345-9cbc-a5933c709313-kube-api-access-lh75d\") pod \"maas-controller-7dc687fcc6-5msks\" (UID: \"e19e8e60-6d2c-4345-9cbc-a5933c709313\") " pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.231691 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.231696    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19e8e60-6d2c-4345-9cbc-a5933c709313-webhook-cert\") pod \"maas-controller-7dc687fcc6-5msks\" (UID: \"e19e8e60-6d2c-4345-9cbc-a5933c709313\") " pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.282449 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.282417    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a11e2dd-41b6-4ea4-94b2-2831457c04c1" path="/var/lib/kubelet/pods/1a11e2dd-41b6-4ea4-94b2-2831457c04c1/volumes"
Jul 29 18:08:00.332353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.332321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh75d\" (UniqueName: \"kubernetes.io/projected/e19e8e60-6d2c-4345-9cbc-a5933c709313-kube-api-access-lh75d\") pod \"maas-controller-7dc687fcc6-5msks\" (UID: \"e19e8e60-6d2c-4345-9cbc-a5933c709313\") " pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.332529 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.332367    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19e8e60-6d2c-4345-9cbc-a5933c709313-webhook-cert\") pod \"maas-controller-7dc687fcc6-5msks\" (UID: \"e19e8e60-6d2c-4345-9cbc-a5933c709313\") " pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.334944 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.334926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19e8e60-6d2c-4345-9cbc-a5933c709313-webhook-cert\") pod \"maas-controller-7dc687fcc6-5msks\" (UID: \"e19e8e60-6d2c-4345-9cbc-a5933c709313\") " pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.340232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.340209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh75d\" (UniqueName: \"kubernetes.io/projected/e19e8e60-6d2c-4345-9cbc-a5933c709313-kube-api-access-lh75d\") pod \"maas-controller-7dc687fcc6-5msks\" (UID: \"e19e8e60-6d2c-4345-9cbc-a5933c709313\") " pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.496566 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.496543    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:00.631579 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:00.631553    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc687fcc6-5msks"]
Jul 29 18:08:00.633359 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:08:00.633328    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19e8e60_6d2c_4345_9cbc_a5933c709313.slice/crio-45ca14d950de8379d248de2bd2f9728ae8561fb7f752f8d4fe3c5cebe4b73d2d WatchSource:0}: Error finding container 45ca14d950de8379d248de2bd2f9728ae8561fb7f752f8d4fe3c5cebe4b73d2d: Status 404 returned error can't find the container with id 45ca14d950de8379d248de2bd2f9728ae8561fb7f752f8d4fe3c5cebe4b73d2d
Jul 29 18:08:01.410569 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:01.410479    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-5msks" event={"ID":"e19e8e60-6d2c-4345-9cbc-a5933c709313","Type":"ContainerStarted","Data":"e0ae88b27e4fb6fb8752c6dff6a72b4a2b2d4d148829f62a90f0d1fbf2651977"}
Jul 29 18:08:01.410569 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:01.410536    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:01.410569 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:01.410555    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc687fcc6-5msks" event={"ID":"e19e8e60-6d2c-4345-9cbc-a5933c709313","Type":"ContainerStarted","Data":"45ca14d950de8379d248de2bd2f9728ae8561fb7f752f8d4fe3c5cebe4b73d2d"}
Jul 29 18:08:01.427285 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:01.427211    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dc687fcc6-5msks" podStartSLOduration=1.071043491 podStartE2EDuration="1.427196049s" totalImagesPullingTime="356.152558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:08:01.424753847 +0000 UTC m=+871.666714884" watchObservedRunningTime="2026-07-29 18:08:01.427196049 +0000 UTC m=+871.669157086"
Jul 29 18:08:12.417962 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:12.417934    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dc687fcc6-5msks"
Jul 29 18:08:30.184607 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:30.184526    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:08:30.185203 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:08:30.185179    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:13:30.216080 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:13:30.216048    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:13:30.217050 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:13:30.217035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:15:00.144938 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.144901    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz"]
Jul 29 18:15:00.148356 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.148340    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz"
Jul 29 18:15:00.150181 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.150162    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:15:00.155016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.154993    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz"]
Jul 29 18:15:00.160190 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.160168    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz"
Jul 29 18:15:00.292555 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.292535    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz"]
Jul 29 18:15:00.295370 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:15:00.295334    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0485e84d_a778_4220_9cf2_7339e9780b15.slice/crio-fcca0a9d91795bf924bc84990c00bbd6d79666e68398f9867cef2a04da8b5e74 WatchSource:0}: Error finding container fcca0a9d91795bf924bc84990c00bbd6d79666e68398f9867cef2a04da8b5e74: Status 404 returned error can't find the container with id fcca0a9d91795bf924bc84990c00bbd6d79666e68398f9867cef2a04da8b5e74
Jul 29 18:15:00.297289 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.297252    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:15:00.403901 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:00.403817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz" event={"ID":"0485e84d-a778-4220-9cf2-7339e9780b15","Type":"ContainerStarted","Data":"fcca0a9d91795bf924bc84990c00bbd6d79666e68398f9867cef2a04da8b5e74"}
Jul 29 18:15:10.460717 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:10.460683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz" event={"ID":"0485e84d-a778-4220-9cf2-7339e9780b15","Type":"ContainerStarted","Data":"d3140981498808a5f2f37e54c2c55cbacc566bf35bea18bab4143fbee3842e1d"}
Jul 29 18:15:10.472911 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:10.472864    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz" podStartSLOduration=0.607333807 podStartE2EDuration="10.472849884s" totalImagesPullingTime="9.865516077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:15:10.472800853 +0000 UTC m=+1300.714761892" watchObservedRunningTime="2026-07-29 18:15:10.472849884 +0000 UTC m=+1300.714810923"
Jul 29 18:15:11.466745 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:11.466648    2560 generic.go:356] "Generic (PLEG): container finished" podID="0485e84d-a778-4220-9cf2-7339e9780b15" containerID="d3140981498808a5f2f37e54c2c55cbacc566bf35bea18bab4143fbee3842e1d" exitCode=0
Jul 29 18:15:11.466745 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:11.466709    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz" event={"ID":"0485e84d-a778-4220-9cf2-7339e9780b15","Type":"ContainerDied","Data":"d3140981498808a5f2f37e54c2c55cbacc566bf35bea18bab4143fbee3842e1d"}
Jul 29 18:15:12.599574 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:12.599547    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz"
Jul 29 18:15:13.476632 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:13.476591    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz"
Jul 29 18:15:13.476814 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:13.476639    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-jkhkz" event={"ID":"0485e84d-a778-4220-9cf2-7339e9780b15","Type":"ContainerDied","Data":"fcca0a9d91795bf924bc84990c00bbd6d79666e68398f9867cef2a04da8b5e74"}
Jul 29 18:15:13.476814 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:15:13.476695    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcca0a9d91795bf924bc84990c00bbd6d79666e68398f9867cef2a04da8b5e74"
Jul 29 18:18:30.253086 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:18:30.253059    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:18:30.253577 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:18:30.253225    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:21:52.972741 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:52.972667    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"]
Jul 29 18:21:52.975195 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:52.973057    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0485e84d-a778-4220-9cf2-7339e9780b15" containerName="cleanup"
Jul 29 18:21:52.975195 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:52.973067    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0485e84d-a778-4220-9cf2-7339e9780b15" containerName="cleanup"
Jul 29 18:21:52.975195 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:52.973135    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0485e84d-a778-4220-9cf2-7339e9780b15" containerName="cleanup"
Jul 29 18:21:52.976170 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:52.976154    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:52.978044 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:52.978026    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a1d6b581-serving-cert\""
Jul 29 18:21:52.987729 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:52.987706    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"]
Jul 29 18:21:53.025421 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.025387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.025552 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.025435    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c71bae0-e277-47e1-9044-a65d9baf8346-openshift-service-ca\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.025552 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.025463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgqj2\" (UniqueName: \"kubernetes.io/projected/9c71bae0-e277-47e1-9044-a65d9baf8346-kube-api-access-jgqj2\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.126931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.126894    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.127146 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.126953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c71bae0-e277-47e1-9044-a65d9baf8346-openshift-service-ca\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.127146 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.126988    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgqj2\" (UniqueName: \"kubernetes.io/projected/9c71bae0-e277-47e1-9044-a65d9baf8346-kube-api-access-jgqj2\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.127146 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:21:53.127056    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-serving-cert: secret "maas-api-e2e-ait-a1d6b581-serving-cert" not found
Jul 29 18:21:53.127146 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:21:53.127124    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls podName:9c71bae0-e277-47e1-9044-a65d9baf8346 nodeName:}" failed. No retries permitted until 2026-07-29 18:21:53.62710456 +0000 UTC m=+1703.869065576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls") pod "maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9" (UID: "9c71bae0-e277-47e1-9044-a65d9baf8346") : secret "maas-api-e2e-ait-a1d6b581-serving-cert" not found
Jul 29 18:21:53.127628 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.127608    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c71bae0-e277-47e1-9044-a65d9baf8346-openshift-service-ca\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.137039 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.137017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgqj2\" (UniqueName: \"kubernetes.io/projected/9c71bae0-e277-47e1-9044-a65d9baf8346-kube-api-access-jgqj2\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.632237 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.632194    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.634769 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.634744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls\") pod \"maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:53.887787 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:53.887693    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:54.028709 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:54.028687    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"]
Jul 29 18:21:54.030446 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:21:54.030408    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c71bae0_e277_47e1_9044_a65d9baf8346.slice/crio-de7fd8b4a6cce9d14513a5e489de1e81231a07fac65dea5ec12e7554c583db8f WatchSource:0}: Error finding container de7fd8b4a6cce9d14513a5e489de1e81231a07fac65dea5ec12e7554c583db8f: Status 404 returned error can't find the container with id de7fd8b4a6cce9d14513a5e489de1e81231a07fac65dea5ec12e7554c583db8f
Jul 29 18:21:54.031961 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:54.031941    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:21:54.379450 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:54.379414    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9" event={"ID":"9c71bae0-e277-47e1-9044-a65d9baf8346","Type":"ContainerStarted","Data":"de7fd8b4a6cce9d14513a5e489de1e81231a07fac65dea5ec12e7554c583db8f"}
Jul 29 18:21:55.385925 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:55.385894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9" event={"ID":"9c71bae0-e277-47e1-9044-a65d9baf8346","Type":"ContainerStarted","Data":"29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa"}
Jul 29 18:21:55.386330 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:55.386027    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:21:55.400699 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:21:55.400642    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9" podStartSLOduration=2.906388093 podStartE2EDuration="3.400625985s" totalImagesPullingTime="494.237892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:21:55.399534774 +0000 UTC m=+1705.641495813" watchObservedRunningTime="2026-07-29 18:21:55.400625985 +0000 UTC m=+1705.642587024"
Jul 29 18:22:01.396583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:01.396546    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:22:17.419914 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.419873    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"]
Jul 29 18:22:17.423600 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.423579    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"
Jul 29 18:22:17.425524 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.425499    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:22:17.430470 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.430446    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"]
Jul 29 18:22:17.556961 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.556929    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b7a9b3-7ffe-4ea1-920a-6b06357c998e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2\" (UID: \"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"
Jul 29 18:22:17.657852 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.657816    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b7a9b3-7ffe-4ea1-920a-6b06357c998e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2\" (UID: \"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"
Jul 29 18:22:17.658517 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.658495    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b7a9b3-7ffe-4ea1-920a-6b06357c998e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2\" (UID: \"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"
Jul 29 18:22:17.736958 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.736928    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"
Jul 29 18:22:17.874934 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:17.874912    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"]
Jul 29 18:22:17.877831 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:22:17.877804    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b7a9b3_7ffe_4ea1_920a_6b06357c998e.slice/crio-4854bb19b2ec9d26ea841acab3fbd74fa5235f3f32a6c25627d9d6efee315286 WatchSource:0}: Error finding container 4854bb19b2ec9d26ea841acab3fbd74fa5235f3f32a6c25627d9d6efee315286: Status 404 returned error can't find the container with id 4854bb19b2ec9d26ea841acab3fbd74fa5235f3f32a6c25627d9d6efee315286
Jul 29 18:22:18.499571 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:18.499538    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2" event={"ID":"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e","Type":"ContainerStarted","Data":"04dd0c79720f151656ac82c90f42e045bc492c075f58ad220f0e66937906d04c"}
Jul 29 18:22:18.499930 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:18.499590    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2" event={"ID":"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e","Type":"ContainerStarted","Data":"4854bb19b2ec9d26ea841acab3fbd74fa5235f3f32a6c25627d9d6efee315286"}
Jul 29 18:22:18.514892 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:18.514830    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2" podStartSLOduration=1.514809512 podStartE2EDuration="1.514809512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:22:18.51293416 +0000 UTC m=+1728.754895200" watchObservedRunningTime="2026-07-29 18:22:18.514809512 +0000 UTC m=+1728.756770561"
Jul 29 18:22:19.506201 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:19.506170    2560 generic.go:356] "Generic (PLEG): container finished" podID="e6b7a9b3-7ffe-4ea1-920a-6b06357c998e" containerID="04dd0c79720f151656ac82c90f42e045bc492c075f58ad220f0e66937906d04c" exitCode=0
Jul 29 18:22:19.506599 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:19.506260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2" event={"ID":"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e","Type":"ContainerDied","Data":"04dd0c79720f151656ac82c90f42e045bc492c075f58ad220f0e66937906d04c"}
Jul 29 18:22:20.642583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:20.642561    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"
Jul 29 18:22:20.788142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:20.788045    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b7a9b3-7ffe-4ea1-920a-6b06357c998e-maas-api-service-ca\") pod \"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e\" (UID: \"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e\") "
Jul 29 18:22:20.788441 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:20.788419    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b7a9b3-7ffe-4ea1-920a-6b06357c998e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6b7a9b3-7ffe-4ea1-920a-6b06357c998e" (UID: "e6b7a9b3-7ffe-4ea1-920a-6b06357c998e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:22:20.889148 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:20.889097    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b7a9b3-7ffe-4ea1-920a-6b06357c998e-maas-api-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:22:21.517610 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:21.517579    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"
Jul 29 18:22:21.517610 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:21.517583    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2" event={"ID":"e6b7a9b3-7ffe-4ea1-920a-6b06357c998e","Type":"ContainerDied","Data":"4854bb19b2ec9d26ea841acab3fbd74fa5235f3f32a6c25627d9d6efee315286"}
Jul 29 18:22:21.517823 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:21.517664    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4854bb19b2ec9d26ea841acab3fbd74fa5235f3f32a6c25627d9d6efee315286"
Jul 29 18:22:23.492192 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.492150    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"]
Jul 29 18:22:23.492685 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.492446    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9" podUID="9c71bae0-e277-47e1-9044-a65d9baf8346" containerName="maas-api" containerID="cri-o://29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa" gracePeriod=30
Jul 29 18:22:23.692829 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.692798    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"]
Jul 29 18:22:23.696027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.696003    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1d6b581-sxhb2"]
Jul 29 18:22:23.745818 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.745755    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:22:23.919603 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.919566    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgqj2\" (UniqueName: \"kubernetes.io/projected/9c71bae0-e277-47e1-9044-a65d9baf8346-kube-api-access-jgqj2\") pod \"9c71bae0-e277-47e1-9044-a65d9baf8346\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") "
Jul 29 18:22:23.919786 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.919681    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c71bae0-e277-47e1-9044-a65d9baf8346-openshift-service-ca\") pod \"9c71bae0-e277-47e1-9044-a65d9baf8346\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") "
Jul 29 18:22:23.919786 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.919757    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls\") pod \"9c71bae0-e277-47e1-9044-a65d9baf8346\" (UID: \"9c71bae0-e277-47e1-9044-a65d9baf8346\") "
Jul 29 18:22:23.920016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.919988    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c71bae0-e277-47e1-9044-a65d9baf8346-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c71bae0-e277-47e1-9044-a65d9baf8346" (UID: "9c71bae0-e277-47e1-9044-a65d9baf8346"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:22:23.921950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.921928    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c71bae0-e277-47e1-9044-a65d9baf8346" (UID: "9c71bae0-e277-47e1-9044-a65d9baf8346"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:22:23.922020 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:23.921950    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c71bae0-e277-47e1-9044-a65d9baf8346-kube-api-access-jgqj2" (OuterVolumeSpecName: "kube-api-access-jgqj2") pod "9c71bae0-e277-47e1-9044-a65d9baf8346" (UID: "9c71bae0-e277-47e1-9044-a65d9baf8346"). InnerVolumeSpecName "kube-api-access-jgqj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:22:24.021200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.021111    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c71bae0-e277-47e1-9044-a65d9baf8346-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:22:24.021200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.021142    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c71bae0-e277-47e1-9044-a65d9baf8346-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:22:24.021200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.021155    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgqj2\" (UniqueName: \"kubernetes.io/projected/9c71bae0-e277-47e1-9044-a65d9baf8346-kube-api-access-jgqj2\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:22:24.282886 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.282804    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b7a9b3-7ffe-4ea1-920a-6b06357c998e" path="/var/lib/kubelet/pods/e6b7a9b3-7ffe-4ea1-920a-6b06357c998e/volumes"
Jul 29 18:22:24.533083 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.532988    2560 generic.go:356] "Generic (PLEG): container finished" podID="9c71bae0-e277-47e1-9044-a65d9baf8346" containerID="29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa" exitCode=0
Jul 29 18:22:24.533083 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.533037    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"
Jul 29 18:22:24.533083 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.533069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9" event={"ID":"9c71bae0-e277-47e1-9044-a65d9baf8346","Type":"ContainerDied","Data":"29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa"}
Jul 29 18:22:24.533622 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.533111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9" event={"ID":"9c71bae0-e277-47e1-9044-a65d9baf8346","Type":"ContainerDied","Data":"de7fd8b4a6cce9d14513a5e489de1e81231a07fac65dea5ec12e7554c583db8f"}
Jul 29 18:22:24.533622 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.533132    2560 scope.go:117] "RemoveContainer" containerID="29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa"
Jul 29 18:22:24.542021 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.541998    2560 scope.go:117] "RemoveContainer" containerID="29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa"
Jul 29 18:22:24.542299 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:22:24.542258    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa\": container with ID starting with 29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa not found: ID does not exist" containerID="29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa"
Jul 29 18:22:24.542403 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.542306    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa"} err="failed to get container status \"29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa\": rpc error: code = NotFound desc = could not find container \"29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa\": container with ID starting with 29bf7d0a984337ed8ac496ccf13bd8e2d016dfcf5d8ae04e1ab9d029f491b7aa not found: ID does not exist"
Jul 29 18:22:24.547708 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.547688    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"]
Jul 29 18:22:24.551964 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.551946    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1d6b581-57999964cc-p6fm9"]
Jul 29 18:22:24.753525 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.753498    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"]
Jul 29 18:22:24.753886 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.753873    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c71bae0-e277-47e1-9044-a65d9baf8346" containerName="maas-api"
Jul 29 18:22:24.753937 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.753888    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c71bae0-e277-47e1-9044-a65d9baf8346" containerName="maas-api"
Jul 29 18:22:24.753937 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.753914    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b7a9b3-7ffe-4ea1-920a-6b06357c998e" containerName="revoke-keys"
Jul 29 18:22:24.753937 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.753919    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b7a9b3-7ffe-4ea1-920a-6b06357c998e" containerName="revoke-keys"
Jul 29 18:22:24.754029 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.753976    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c71bae0-e277-47e1-9044-a65d9baf8346" containerName="maas-api"
Jul 29 18:22:24.754029 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.753983    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b7a9b3-7ffe-4ea1-920a-6b06357c998e" containerName="revoke-keys"
Jul 29 18:22:24.758493 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.758475    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.760479 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.760457    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:22:24.760583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.760532    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-38933e69-gw-openshift-default-dockercfg-2fn7h\""
Jul 29 18:22:24.771295 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.771248    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"]
Jul 29 18:22:24.931868 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.931766    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-socket\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.931868 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.931815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-token\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.931868 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.931846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-envoy\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.932121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.931888    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be00d64-5165-4de2-8a44-ed88c7aece7d-istiod-ca-cert\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.932121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.931958    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-data\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.932121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.932014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-podinfo\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.932121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.932034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvdcs\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-kube-api-access-cvdcs\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.932121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.932054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-certs\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:24.932349 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:24.932143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-credential-socket\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033218 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-credential-socket\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-socket\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033262    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-token\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033323    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-envoy\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033367    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be00d64-5165-4de2-8a44-ed88c7aece7d-istiod-ca-cert\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033401    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-data\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033446    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-podinfo\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033472    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvdcs\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-kube-api-access-cvdcs\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033502    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-certs\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033655    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-credential-socket\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033681    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-socket\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033966 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033896    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-certs\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.033966 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.033914    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-data\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.034086 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.034067    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be00d64-5165-4de2-8a44-ed88c7aece7d-istiod-ca-cert\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.036347 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.036325    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-envoy\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.036654 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.036631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-podinfo\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.040964 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.040941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvdcs\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-kube-api-access-cvdcs\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.040964 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.040956    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-token\") pod \"e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") " pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.076209 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.076181    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:25.217805 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.217781    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"]
Jul 29 18:22:25.220057 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:22:25.220030    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be00d64_5165_4de2_8a44_ed88c7aece7d.slice/crio-96ecffb16f3b5e1e5d7838bd04bdf514acaf02328da93a52ff7b04b4a8e82cb7 WatchSource:0}: Error finding container 96ecffb16f3b5e1e5d7838bd04bdf514acaf02328da93a52ff7b04b4a8e82cb7: Status 404 returned error can't find the container with id 96ecffb16f3b5e1e5d7838bd04bdf514acaf02328da93a52ff7b04b4a8e82cb7
Jul 29 18:22:25.538530 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:25.538495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" event={"ID":"2be00d64-5165-4de2-8a44-ed88c7aece7d","Type":"ContainerStarted","Data":"96ecffb16f3b5e1e5d7838bd04bdf514acaf02328da93a52ff7b04b4a8e82cb7"}
Jul 29 18:22:26.281641 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:26.281606    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c71bae0-e277-47e1-9044-a65d9baf8346" path="/var/lib/kubelet/pods/9c71bae0-e277-47e1-9044-a65d9baf8346/volumes"
Jul 29 18:22:28.244660 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:28.244623    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:22:28.244940 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:28.244692    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:22:28.244940 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:28.244720    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:22:28.556876 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:28.556783    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" event={"ID":"2be00d64-5165-4de2-8a44-ed88c7aece7d","Type":"ContainerStarted","Data":"d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c"}
Jul 29 18:22:28.578874 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:28.578813    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" podStartSLOduration=1.5565651630000001 podStartE2EDuration="4.578796674s" totalImagesPullingTime="3.022231511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:22:28.577929921 +0000 UTC m=+1738.819890972" watchObservedRunningTime="2026-07-29 18:22:28.578796674 +0000 UTC m=+1738.820757712"
Jul 29 18:22:29.077190 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:29.077147    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:29.082074 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:29.082051    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:29.561645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:29.561615    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:29.562583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:29.562563    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:30.183095 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.183062    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"]
Jul 29 18:22:30.186697 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.186679    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.188562 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.188546    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-38933e69-serving-cert\""
Jul 29 18:22:30.194825 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.194807    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"]
Jul 29 18:22:30.285651 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.285615    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.285821 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.285685    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd6jh\" (UniqueName: \"kubernetes.io/projected/86f071b4-5fe0-463a-ab5d-c10d28f23933-kube-api-access-zd6jh\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.285821 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.285768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f071b4-5fe0-463a-ab5d-c10d28f23933-openshift-service-ca\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.386945 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.386909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f071b4-5fe0-463a-ab5d-c10d28f23933-openshift-service-ca\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.387135 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.387020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.387135 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.387051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd6jh\" (UniqueName: \"kubernetes.io/projected/86f071b4-5fe0-463a-ab5d-c10d28f23933-kube-api-access-zd6jh\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.387254 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:22:30.387170    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-serving-cert: secret "maas-api-e2e-migrate-38933e69-serving-cert" not found
Jul 29 18:22:30.387350 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:22:30.387259    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls podName:86f071b4-5fe0-463a-ab5d-c10d28f23933 nodeName:}" failed. No retries permitted until 2026-07-29 18:22:30.887235163 +0000 UTC m=+1741.129196198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls") pod "maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq" (UID: "86f071b4-5fe0-463a-ab5d-c10d28f23933") : secret "maas-api-e2e-migrate-38933e69-serving-cert" not found
Jul 29 18:22:30.387649 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.387630    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f071b4-5fe0-463a-ab5d-c10d28f23933-openshift-service-ca\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.396661 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.396643    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd6jh\" (UniqueName: \"kubernetes.io/projected/86f071b4-5fe0-463a-ab5d-c10d28f23933-kube-api-access-zd6jh\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.892789 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.892752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:30.895316 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:30.895287    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls\") pod \"maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:31.097468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:31.097428    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:31.240028 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:31.240000    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"]
Jul 29 18:22:31.242107 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:22:31.242079    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f071b4_5fe0_463a_ab5d_c10d28f23933.slice/crio-41671d464985ba7edbb6cf7760cf3be95cf070bf5dcb071ee63327ccf8ea69a8 WatchSource:0}: Error finding container 41671d464985ba7edbb6cf7760cf3be95cf070bf5dcb071ee63327ccf8ea69a8: Status 404 returned error can't find the container with id 41671d464985ba7edbb6cf7760cf3be95cf070bf5dcb071ee63327ccf8ea69a8
Jul 29 18:22:31.572552 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:31.572516    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq" event={"ID":"86f071b4-5fe0-463a-ab5d-c10d28f23933","Type":"ContainerStarted","Data":"41671d464985ba7edbb6cf7760cf3be95cf070bf5dcb071ee63327ccf8ea69a8"}
Jul 29 18:22:32.578954 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:32.578919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq" event={"ID":"86f071b4-5fe0-463a-ab5d-c10d28f23933","Type":"ContainerStarted","Data":"dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f"}
Jul 29 18:22:32.579357 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:32.578987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:32.594375 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:32.594313    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq" podStartSLOduration=2.172333369 podStartE2EDuration="2.59429255s" totalImagesPullingTime="421.959181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:22:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:22:32.593705956 +0000 UTC m=+1742.835666993" watchObservedRunningTime="2026-07-29 18:22:32.59429255 +0000 UTC m=+1742.836253583"
Jul 29 18:22:38.590288 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:38.590235    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:22:42.630716 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:42.630680    2560 generic.go:356] "Generic (PLEG): container finished" podID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerID="d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c" exitCode=0
Jul 29 18:22:42.631149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:42.630758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" event={"ID":"2be00d64-5165-4de2-8a44-ed88c7aece7d","Type":"ContainerDied","Data":"d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c"}
Jul 29 18:22:42.631212 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:42.631197    2560 scope.go:117] "RemoveContainer" containerID="d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c"
Jul 29 18:22:42.632295 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:42.632243    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:22:42.632358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:42.632340    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:22:42.632399 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:42.632371    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:22:43.641390 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:43.641354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" event={"ID":"2be00d64-5165-4de2-8a44-ed88c7aece7d","Type":"ContainerStarted","Data":"af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d"}
Jul 29 18:22:44.077419 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:44.077380    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:44.078525 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:44.078494    2560 patch_prober.go:28] interesting pod/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:22:44.078665 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:44.078574    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:22:44.562408 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:44.562351    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:45.077470 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:45.077433    2560 patch_prober.go:28] interesting pod/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:22:45.077856 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:45.077507    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:22:46.077076 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:46.077038    2560 patch_prober.go:28] interesting pod/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:22:46.077253 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:46.077101    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:22:47.077610 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:47.077574    2560 patch_prober.go:28] interesting pod/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:22:47.078000 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:47.077628    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:22:48.081038 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:48.081012    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:48.081985 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:48.081968    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:22:55.013539 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.013503    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"]
Jul 29 18:22:55.020373 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.020352    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"
Jul 29 18:22:55.022611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.022585    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:22:55.026458 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.026430    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"]
Jul 29 18:22:55.135066 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.135034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50626179-4e29-4562-a25a-cc7227d97ded-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-38933e69-f6km2\" (UID: \"50626179-4e29-4562-a25a-cc7227d97ded\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"
Jul 29 18:22:55.236207 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.236171    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50626179-4e29-4562-a25a-cc7227d97ded-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-38933e69-f6km2\" (UID: \"50626179-4e29-4562-a25a-cc7227d97ded\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"
Jul 29 18:22:55.236940 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.236923    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50626179-4e29-4562-a25a-cc7227d97ded-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-38933e69-f6km2\" (UID: \"50626179-4e29-4562-a25a-cc7227d97ded\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"
Jul 29 18:22:55.333532 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.333396    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"
Jul 29 18:22:55.478321 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.478299    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"]
Jul 29 18:22:55.481079 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:22:55.481051    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50626179_4e29_4562_a25a_cc7227d97ded.slice/crio-763f88aa7ff7b3b8aabec36bef56c34e4e13d31c4e970cd7262a6dd5a0a17947 WatchSource:0}: Error finding container 763f88aa7ff7b3b8aabec36bef56c34e4e13d31c4e970cd7262a6dd5a0a17947: Status 404 returned error can't find the container with id 763f88aa7ff7b3b8aabec36bef56c34e4e13d31c4e970cd7262a6dd5a0a17947
Jul 29 18:22:55.703751 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:55.703653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2" event={"ID":"50626179-4e29-4562-a25a-cc7227d97ded","Type":"ContainerStarted","Data":"763f88aa7ff7b3b8aabec36bef56c34e4e13d31c4e970cd7262a6dd5a0a17947"}
Jul 29 18:22:56.710891 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:56.710862    2560 generic.go:356] "Generic (PLEG): container finished" podID="50626179-4e29-4562-a25a-cc7227d97ded" containerID="b065b8c82c63ed5b05634c0c70b6482fbce5f972052b2c0cad2a098a6c26e24b" exitCode=0
Jul 29 18:22:56.711294 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:56.710909    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2" event={"ID":"50626179-4e29-4562-a25a-cc7227d97ded","Type":"ContainerDied","Data":"b065b8c82c63ed5b05634c0c70b6482fbce5f972052b2c0cad2a098a6c26e24b"}
Jul 29 18:22:57.854447 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:57.854419    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"
Jul 29 18:22:57.964908 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:57.964873    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50626179-4e29-4562-a25a-cc7227d97ded-maas-api-service-ca\") pod \"50626179-4e29-4562-a25a-cc7227d97ded\" (UID: \"50626179-4e29-4562-a25a-cc7227d97ded\") "
Jul 29 18:22:57.965225 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:57.965202    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50626179-4e29-4562-a25a-cc7227d97ded-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50626179-4e29-4562-a25a-cc7227d97ded" (UID: "50626179-4e29-4562-a25a-cc7227d97ded"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:22:58.066529 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:58.066499    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50626179-4e29-4562-a25a-cc7227d97ded-maas-api-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:22:58.723453 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:58.723424    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"
Jul 29 18:22:58.723624 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:58.723426    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2" event={"ID":"50626179-4e29-4562-a25a-cc7227d97ded","Type":"ContainerDied","Data":"763f88aa7ff7b3b8aabec36bef56c34e4e13d31c4e970cd7262a6dd5a0a17947"}
Jul 29 18:22:58.723624 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:22:58.723555    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="763f88aa7ff7b3b8aabec36bef56c34e4e13d31c4e970cd7262a6dd5a0a17947"
Jul 29 18:23:01.429734 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.429696    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"]
Jul 29 18:23:01.430225 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.430024    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq" podUID="86f071b4-5fe0-463a-ab5d-c10d28f23933" containerName="maas-api" containerID="cri-o://dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f" gracePeriod=30
Jul 29 18:23:01.628328 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.628285    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"]
Jul 29 18:23:01.630185 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.630157    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-38933e69-f6km2"]
Jul 29 18:23:01.684344 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.684278    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:23:01.743000 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.742972    2560 generic.go:356] "Generic (PLEG): container finished" podID="86f071b4-5fe0-463a-ab5d-c10d28f23933" containerID="dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f" exitCode=0
Jul 29 18:23:01.743193 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.743037    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"
Jul 29 18:23:01.743193 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.743080    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq" event={"ID":"86f071b4-5fe0-463a-ab5d-c10d28f23933","Type":"ContainerDied","Data":"dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f"}
Jul 29 18:23:01.743193 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.743162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq" event={"ID":"86f071b4-5fe0-463a-ab5d-c10d28f23933","Type":"ContainerDied","Data":"41671d464985ba7edbb6cf7760cf3be95cf070bf5dcb071ee63327ccf8ea69a8"}
Jul 29 18:23:01.743397 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.743197    2560 scope.go:117] "RemoveContainer" containerID="dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f"
Jul 29 18:23:01.753863 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.753839    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"]
Jul 29 18:23:01.754055 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.754035    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy" containerID="cri-o://af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d" gracePeriod=30
Jul 29 18:23:01.755392 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.755370    2560 scope.go:117] "RemoveContainer" containerID="dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f"
Jul 29 18:23:01.755685 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:01.755659    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f\": container with ID starting with dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f not found: ID does not exist" containerID="dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f"
Jul 29 18:23:01.755766 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.755698    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f"} err="failed to get container status \"dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f\": rpc error: code = NotFound desc = could not find container \"dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f\": container with ID starting with dfbd9e87267bd9d48d475d30fc45621847b9bb4bdf6971747bf7e98ddbc2c28f not found: ID does not exist"
Jul 29 18:23:01.800415 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.800388    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd6jh\" (UniqueName: \"kubernetes.io/projected/86f071b4-5fe0-463a-ab5d-c10d28f23933-kube-api-access-zd6jh\") pod \"86f071b4-5fe0-463a-ab5d-c10d28f23933\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") "
Jul 29 18:23:01.800618 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.800456    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls\") pod \"86f071b4-5fe0-463a-ab5d-c10d28f23933\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") "
Jul 29 18:23:01.800618 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.800477    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f071b4-5fe0-463a-ab5d-c10d28f23933-openshift-service-ca\") pod \"86f071b4-5fe0-463a-ab5d-c10d28f23933\" (UID: \"86f071b4-5fe0-463a-ab5d-c10d28f23933\") "
Jul 29 18:23:01.800846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.800818    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f071b4-5fe0-463a-ab5d-c10d28f23933-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86f071b4-5fe0-463a-ab5d-c10d28f23933" (UID: "86f071b4-5fe0-463a-ab5d-c10d28f23933"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:23:01.802763 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.802726    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86f071b4-5fe0-463a-ab5d-c10d28f23933" (UID: "86f071b4-5fe0-463a-ab5d-c10d28f23933"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:23:01.802763 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.802739    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f071b4-5fe0-463a-ab5d-c10d28f23933-kube-api-access-zd6jh" (OuterVolumeSpecName: "kube-api-access-zd6jh") pod "86f071b4-5fe0-463a-ab5d-c10d28f23933" (UID: "86f071b4-5fe0-463a-ab5d-c10d28f23933"). InnerVolumeSpecName "kube-api-access-zd6jh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:23:01.901168 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.901132    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd6jh\" (UniqueName: \"kubernetes.io/projected/86f071b4-5fe0-463a-ab5d-c10d28f23933-kube-api-access-zd6jh\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:01.901168 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.901160    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f071b4-5fe0-463a-ab5d-c10d28f23933-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:01.901168 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:01.901169    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f071b4-5fe0-463a-ab5d-c10d28f23933-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:02.066141 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.066109    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"]
Jul 29 18:23:02.070327 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.070302    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-38933e69-7654fb65b4-pprfq"]
Jul 29 18:23:02.285292 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.283406    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50626179-4e29-4562-a25a-cc7227d97ded" path="/var/lib/kubelet/pods/50626179-4e29-4562-a25a-cc7227d97ded/volumes"
Jul 29 18:23:02.285292 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.284085    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86f071b4-5fe0-463a-ab5d-c10d28f23933" path="/var/lib/kubelet/pods/86f071b4-5fe0-463a-ab5d-c10d28f23933/volumes"
Jul 29 18:23:02.495573 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.495545    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"]
Jul 29 18:23:02.495950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.495930    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50626179-4e29-4562-a25a-cc7227d97ded" containerName="revoke-keys"
Jul 29 18:23:02.495950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.495942    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="50626179-4e29-4562-a25a-cc7227d97ded" containerName="revoke-keys"
Jul 29 18:23:02.496023 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.495963    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86f071b4-5fe0-463a-ab5d-c10d28f23933" containerName="maas-api"
Jul 29 18:23:02.496023 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.495969    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="86f071b4-5fe0-463a-ab5d-c10d28f23933" containerName="maas-api"
Jul 29 18:23:02.496083 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.496023    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="86f071b4-5fe0-463a-ab5d-c10d28f23933" containerName="maas-api"
Jul 29 18:23:02.496083 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.496034    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="50626179-4e29-4562-a25a-cc7227d97ded" containerName="revoke-keys"
Jul 29 18:23:02.500332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.500311    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.502235 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.502217    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-697aa257-openshift-default-dockercfg-hmdtf\""
Jul 29 18:23:02.512724 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.512694    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"]
Jul 29 18:23:02.608339 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608309    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-socket\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608538 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-token\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608538 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-podinfo\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608538 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608467    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-credential-socket\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608715 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-certs\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608715 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc2jw\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-kube-api-access-dc2jw\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608715 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608646    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-data\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608715 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608701    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-envoy\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.608880 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.608730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccc8b27e-9475-48f3-a202-4bba216e8282-istiod-ca-cert\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.709879 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.709841    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-podinfo\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.709879 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.709883    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-credential-socket\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710128 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.709923    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-certs\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710128 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.709951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc2jw\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-kube-api-access-dc2jw\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710128 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.709969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-data\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710128 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.709992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-envoy\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710356 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710215    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccc8b27e-9475-48f3-a202-4bba216e8282-istiod-ca-cert\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710356 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-socket\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710467 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710395    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-certs\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710467 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710411    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-token\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710570 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710470    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-credential-socket\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710570 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710492    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-data\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-socket\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.710971 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.710944    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccc8b27e-9475-48f3-a202-4bba216e8282-istiod-ca-cert\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.712451 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.712428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-envoy\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.712659 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.712636    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-podinfo\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.717676 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.717651    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-token\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.717893 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.717879    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc2jw\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-kube-api-access-dc2jw\") pod \"e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") " pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.811060 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.810967    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:02.948636 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.948607    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"]
Jul 29 18:23:02.950677 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:23:02.950649    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc8b27e_9475_48f3_a202_4bba216e8282.slice/crio-9dfda2f0bbdf5e4e4eb1705decfd755fcbc6d080b54bae56c974ad1f27ff69c3 WatchSource:0}: Error finding container 9dfda2f0bbdf5e4e4eb1705decfd755fcbc6d080b54bae56c974ad1f27ff69c3: Status 404 returned error can't find the container with id 9dfda2f0bbdf5e4e4eb1705decfd755fcbc6d080b54bae56c974ad1f27ff69c3
Jul 29 18:23:02.952922 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.952892    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:23:02.952996 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.952958    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:23:02.953035 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:02.952994    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:23:03.084308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:03.084191    2560 patch_prober.go:28] interesting pod/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 29 18:23:03.084308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:03.084290    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:23:03.756500 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:03.756459    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" event={"ID":"ccc8b27e-9475-48f3-a202-4bba216e8282","Type":"ContainerStarted","Data":"92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652"}
Jul 29 18:23:03.756889 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:03.756524    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" event={"ID":"ccc8b27e-9475-48f3-a202-4bba216e8282","Type":"ContainerStarted","Data":"9dfda2f0bbdf5e4e4eb1705decfd755fcbc6d080b54bae56c974ad1f27ff69c3"}
Jul 29 18:23:03.775593 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:03.775543    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" podStartSLOduration=1.775525985 podStartE2EDuration="1.775525985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:23:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:23:03.77437156 +0000 UTC m=+1774.016332597" watchObservedRunningTime="2026-07-29 18:23:03.775525985 +0000 UTC m=+1774.017487022"
Jul 29 18:23:03.811813 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:03.811777    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:04.816516 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:04.816488    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:05.770625 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:05.770600    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:05.771536 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:05.771518    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:06.201642 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.201549    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"]
Jul 29 18:23:06.205258 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.205239    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.207069 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.207049    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-697aa257-serving-cert\""
Jul 29 18:23:06.215915 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.215878    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"]
Jul 29 18:23:06.346286 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.346243    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.346467 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.346451    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1964218b-cfbd-4806-89f4-26596d3e8144-openshift-service-ca\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.346527 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.346509    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfj2s\" (UniqueName: \"kubernetes.io/projected/1964218b-cfbd-4806-89f4-26596d3e8144-kube-api-access-vfj2s\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.447718 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.447684    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.447933 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.447768    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1964218b-cfbd-4806-89f4-26596d3e8144-openshift-service-ca\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.447933 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.447807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfj2s\" (UniqueName: \"kubernetes.io/projected/1964218b-cfbd-4806-89f4-26596d3e8144-kube-api-access-vfj2s\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.447933 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:06.447838    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-serving-cert: secret "maas-api-e2e-ait-697aa257-serving-cert" not found
Jul 29 18:23:06.447933 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:06.447915    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls podName:1964218b-cfbd-4806-89f4-26596d3e8144 nodeName:}" failed. No retries permitted until 2026-07-29 18:23:06.947894971 +0000 UTC m=+1777.189855989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls") pod "maas-api-e2e-ait-697aa257-7587d889c7-7lgtf" (UID: "1964218b-cfbd-4806-89f4-26596d3e8144") : secret "maas-api-e2e-ait-697aa257-serving-cert" not found
Jul 29 18:23:06.448458 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.448441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1964218b-cfbd-4806-89f4-26596d3e8144-openshift-service-ca\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.465495 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.465421    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfj2s\" (UniqueName: \"kubernetes.io/projected/1964218b-cfbd-4806-89f4-26596d3e8144-kube-api-access-vfj2s\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.953856 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.953825    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:06.956584 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:06.956556    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls\") pod \"maas-api-e2e-ait-697aa257-7587d889c7-7lgtf\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:07.019728 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.019654    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:23:07.117569 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.117534    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:07.155532 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155504    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-envoy\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155547    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-certs\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155579    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-token\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155622    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-credential-socket\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155702 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155662    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-data\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155711    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-socket\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155731    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-podinfo\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155782    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be00d64-5165-4de2-8a44-ed88c7aece7d-istiod-ca-cert\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.155943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155834    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvdcs\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-kube-api-access-cvdcs\") pod \"2be00d64-5165-4de2-8a44-ed88c7aece7d\" (UID: \"2be00d64-5165-4de2-8a44-ed88c7aece7d\") "
Jul 29 18:23:07.156148 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.155978    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:07.156219 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.156191    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.156412 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.156384    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:07.156730 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.156704    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2be00d64-5165-4de2-8a44-ed88c7aece7d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:23:07.156832 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.156706    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:07.156922 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.156884    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-data" (OuterVolumeSpecName: "istio-data") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:07.158219 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.158191    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-token" (OuterVolumeSpecName: "istio-token") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:23:07.158372 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.158190    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:23:07.158643 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.158620    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:07.158643 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.158627    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-kube-api-access-cvdcs" (OuterVolumeSpecName: "kube-api-access-cvdcs") pod "2be00d64-5165-4de2-8a44-ed88c7aece7d" (UID: "2be00d64-5165-4de2-8a44-ed88c7aece7d"). InnerVolumeSpecName "kube-api-access-cvdcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:23:07.256714 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256682    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.256714 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256708    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2be00d64-5165-4de2-8a44-ed88c7aece7d-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.256714 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256718    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvdcs\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-kube-api-access-cvdcs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.257149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256726    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.257149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256735    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.257149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256742    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.257149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256750    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.257149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.256758    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2be00d64-5165-4de2-8a44-ed88c7aece7d-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:07.261119 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.261097    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"]
Jul 29 18:23:07.263583 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:23:07.263553    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1964218b_cfbd_4806_89f4_26596d3e8144.slice/crio-3683fb24ac16fd55d7bb4ae2cc9cfe69aad1e873f2a1170b9cb395729545347b WatchSource:0}: Error finding container 3683fb24ac16fd55d7bb4ae2cc9cfe69aad1e873f2a1170b9cb395729545347b: Status 404 returned error can't find the container with id 3683fb24ac16fd55d7bb4ae2cc9cfe69aad1e873f2a1170b9cb395729545347b
Jul 29 18:23:07.783930 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.783905    2560 generic.go:356] "Generic (PLEG): container finished" podID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerID="af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d" exitCode=0
Jul 29 18:23:07.784054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.783947    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"
Jul 29 18:23:07.784054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.783991    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" event={"ID":"2be00d64-5165-4de2-8a44-ed88c7aece7d","Type":"ContainerDied","Data":"af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d"}
Jul 29 18:23:07.784054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.784046    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq" event={"ID":"2be00d64-5165-4de2-8a44-ed88c7aece7d","Type":"ContainerDied","Data":"96ecffb16f3b5e1e5d7838bd04bdf514acaf02328da93a52ff7b04b4a8e82cb7"}
Jul 29 18:23:07.784229 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.784077    2560 scope.go:117] "RemoveContainer" containerID="af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d"
Jul 29 18:23:07.785598 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.785541    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf" event={"ID":"1964218b-cfbd-4806-89f4-26596d3e8144","Type":"ContainerStarted","Data":"e5c497d923ed55d6ccc9d2ae314f330ba207032f99dcc0e729c4fb76ccb5dd38"}
Jul 29 18:23:07.785690 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.785625    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf" event={"ID":"1964218b-cfbd-4806-89f4-26596d3e8144","Type":"ContainerStarted","Data":"3683fb24ac16fd55d7bb4ae2cc9cfe69aad1e873f2a1170b9cb395729545347b"}
Jul 29 18:23:07.793950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.793934    2560 scope.go:117] "RemoveContainer" containerID="d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c"
Jul 29 18:23:07.802113 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.802072    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf" podStartSLOduration=1.360225769 podStartE2EDuration="1.802057108s" totalImagesPullingTime="441.831339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:23:07.801456112 +0000 UTC m=+1778.043417151" watchObservedRunningTime="2026-07-29 18:23:07.802057108 +0000 UTC m=+1778.044018145"
Jul 29 18:23:07.802654 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.802637    2560 scope.go:117] "RemoveContainer" containerID="af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d"
Jul 29 18:23:07.802978 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:07.802958    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d\": container with ID starting with af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d not found: ID does not exist" containerID="af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d"
Jul 29 18:23:07.803037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.802987    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d"} err="failed to get container status \"af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d\": rpc error: code = NotFound desc = could not find container \"af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d\": container with ID starting with af1362369cb7f491b5e832ae86d4132cc39830da6a08d028ff5195a5645fd80d not found: ID does not exist"
Jul 29 18:23:07.803037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.803003    2560 scope.go:117] "RemoveContainer" containerID="d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c"
Jul 29 18:23:07.803226 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:07.803209    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c\": container with ID starting with d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c not found: ID does not exist" containerID="d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c"
Jul 29 18:23:07.803298 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.803230    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c"} err="failed to get container status \"d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c\": rpc error: code = NotFound desc = could not find container \"d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c\": container with ID starting with d7611f47df7b4fd8b907cb33474c370c3ead4600be538b5d047ac64400564f4c not found: ID does not exist"
Jul 29 18:23:07.814236 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.814216    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"]
Jul 29 18:23:07.818630 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:07.818607    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-38933e69-gw-openshift-default-f774b49fb-mjgcq"]
Jul 29 18:23:08.281779 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:08.281743    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" path="/var/lib/kubelet/pods/2be00d64-5165-4de2-8a44-ed88c7aece7d/volumes"
Jul 29 18:23:08.791428 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:08.791403    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:14.801530 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:14.801503    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:20.772343 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.772301    2560 patch_prober.go:28] interesting pod/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 29 18:23:20.772742 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.772374    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 29 18:23:20.868879 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.868800    2560 generic.go:356] "Generic (PLEG): container finished" podID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerID="92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652" exitCode=0
Jul 29 18:23:20.868879 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.868869    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" event={"ID":"ccc8b27e-9475-48f3-a202-4bba216e8282","Type":"ContainerDied","Data":"92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652"}
Jul 29 18:23:20.869306 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.869290    2560 scope.go:117] "RemoveContainer" containerID="92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652"
Jul 29 18:23:20.870405 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.870373    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:23:20.870534 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.870479    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:23:20.870534 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:20.870529    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:23:21.876001 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:21.875968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" event={"ID":"ccc8b27e-9475-48f3-a202-4bba216e8282","Type":"ContainerStarted","Data":"36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618"}
Jul 29 18:23:22.811770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:22.811723    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:22.812178 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:22.812141    2560 patch_prober.go:28] interesting pod/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 29 18:23:22.812348 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:22.812235    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 29 18:23:23.811723 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:23.811674    2560 patch_prober.go:28] interesting pod/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 29 18:23:23.812223 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:23.811762    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 29 18:23:24.811744 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:24.811709    2560 patch_prober.go:28] interesting pod/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 29 18:23:24.812353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:24.812320    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 29 18:23:25.815810 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:25.815779    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:25.816249 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:25.816049    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:25.816772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:25.816755    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:30.286769 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:30.286737    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:23:30.289010 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:30.288989    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:23:46.750907 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:46.750868    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"]
Jul 29 18:23:46.751495 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:46.751160    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf" podUID="1964218b-cfbd-4806-89f4-26596d3e8144" containerName="maas-api" containerID="cri-o://e5c497d923ed55d6ccc9d2ae314f330ba207032f99dcc0e729c4fb76ccb5dd38" gracePeriod=30
Jul 29 18:23:47.006384 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.006226    2560 generic.go:356] "Generic (PLEG): container finished" podID="1964218b-cfbd-4806-89f4-26596d3e8144" containerID="e5c497d923ed55d6ccc9d2ae314f330ba207032f99dcc0e729c4fb76ccb5dd38" exitCode=0
Jul 29 18:23:47.006384 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.006301    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf" event={"ID":"1964218b-cfbd-4806-89f4-26596d3e8144","Type":"ContainerDied","Data":"e5c497d923ed55d6ccc9d2ae314f330ba207032f99dcc0e729c4fb76ccb5dd38"}
Jul 29 18:23:47.006384 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.006361    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf" event={"ID":"1964218b-cfbd-4806-89f4-26596d3e8144","Type":"ContainerDied","Data":"3683fb24ac16fd55d7bb4ae2cc9cfe69aad1e873f2a1170b9cb395729545347b"}
Jul 29 18:23:47.006384 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.006381    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3683fb24ac16fd55d7bb4ae2cc9cfe69aad1e873f2a1170b9cb395729545347b"
Jul 29 18:23:47.006621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.006502    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:47.161143 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.161089    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1964218b-cfbd-4806-89f4-26596d3e8144-openshift-service-ca\") pod \"1964218b-cfbd-4806-89f4-26596d3e8144\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") "
Jul 29 18:23:47.161353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.161222    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfj2s\" (UniqueName: \"kubernetes.io/projected/1964218b-cfbd-4806-89f4-26596d3e8144-kube-api-access-vfj2s\") pod \"1964218b-cfbd-4806-89f4-26596d3e8144\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") "
Jul 29 18:23:47.161353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.161257    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls\") pod \"1964218b-cfbd-4806-89f4-26596d3e8144\" (UID: \"1964218b-cfbd-4806-89f4-26596d3e8144\") "
Jul 29 18:23:47.161563 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.161531    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1964218b-cfbd-4806-89f4-26596d3e8144-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1964218b-cfbd-4806-89f4-26596d3e8144" (UID: "1964218b-cfbd-4806-89f4-26596d3e8144"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:23:47.161707 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.161647    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1964218b-cfbd-4806-89f4-26596d3e8144-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:47.163565 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.163540    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1964218b-cfbd-4806-89f4-26596d3e8144-kube-api-access-vfj2s" (OuterVolumeSpecName: "kube-api-access-vfj2s") pod "1964218b-cfbd-4806-89f4-26596d3e8144" (UID: "1964218b-cfbd-4806-89f4-26596d3e8144"). InnerVolumeSpecName "kube-api-access-vfj2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:23:47.163669 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.163646    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1964218b-cfbd-4806-89f4-26596d3e8144" (UID: "1964218b-cfbd-4806-89f4-26596d3e8144"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:23:47.262607 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.262513    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfj2s\" (UniqueName: \"kubernetes.io/projected/1964218b-cfbd-4806-89f4-26596d3e8144-kube-api-access-vfj2s\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:47.262607 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:47.262552    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1964218b-cfbd-4806-89f4-26596d3e8144-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:48.011610 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:48.011576    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"
Jul 29 18:23:48.031202 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:48.031174    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"]
Jul 29 18:23:48.039798 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:48.039769    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-697aa257-7587d889c7-7lgtf"]
Jul 29 18:23:48.127572 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:48.127542    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"]
Jul 29 18:23:48.127789 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:48.127767    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy" containerID="cri-o://36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618" gracePeriod=30
Jul 29 18:23:48.282258 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:48.282177    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1964218b-cfbd-4806-89f4-26596d3e8144" path="/var/lib/kubelet/pods/1964218b-cfbd-4806-89f4-26596d3e8144/volumes"
Jul 29 18:23:52.653693 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.653661    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"]
Jul 29 18:23:52.654388 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.654347    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy"
Jul 29 18:23:52.654388 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.654372    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy"
Jul 29 18:23:52.654388 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.654384    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1964218b-cfbd-4806-89f4-26596d3e8144" containerName="maas-api"
Jul 29 18:23:52.654603 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.654393    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1964218b-cfbd-4806-89f4-26596d3e8144" containerName="maas-api"
Jul 29 18:23:52.654603 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.654516    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1964218b-cfbd-4806-89f4-26596d3e8144" containerName="maas-api"
Jul 29 18:23:52.654603 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.654533    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy"
Jul 29 18:23:52.654603 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.654543    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy"
Jul 29 18:23:52.658054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.658032    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.660666 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.660639    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0eed4609-serving-cert\""
Jul 29 18:23:52.678955 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.678931    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"]
Jul 29 18:23:52.711704 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.711671    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtr7n\" (UniqueName: \"kubernetes.io/projected/70c9f051-3d50-4345-93e5-71333abcd4ca-kube-api-access-wtr7n\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.711874 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.711734    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c9f051-3d50-4345-93e5-71333abcd4ca-openshift-service-ca\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.711874 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.711798    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.812549 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.812517    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtr7n\" (UniqueName: \"kubernetes.io/projected/70c9f051-3d50-4345-93e5-71333abcd4ca-kube-api-access-wtr7n\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.812724 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.812590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c9f051-3d50-4345-93e5-71333abcd4ca-openshift-service-ca\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.812724 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.812628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.812804 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:52.812733    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-serving-cert: secret "maas-api-e2e-derive-0eed4609-serving-cert" not found
Jul 29 18:23:52.812804 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:52.812790    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls podName:70c9f051-3d50-4345-93e5-71333abcd4ca nodeName:}" failed. No retries permitted until 2026-07-29 18:23:53.312771091 +0000 UTC m=+1823.554732108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls") pod "maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8" (UID: "70c9f051-3d50-4345-93e5-71333abcd4ca") : secret "maas-api-e2e-derive-0eed4609-serving-cert" not found
Jul 29 18:23:52.813251 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.813231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c9f051-3d50-4345-93e5-71333abcd4ca-openshift-service-ca\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:52.822085 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:52.822052    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtr7n\" (UniqueName: \"kubernetes.io/projected/70c9f051-3d50-4345-93e5-71333abcd4ca-kube-api-access-wtr7n\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:53.319064 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.319029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:53.321712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.321686    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls\") pod \"maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:53.383765 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.383738    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:53.419556 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419473    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-podinfo\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.419556 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419515    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-token\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.419772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419594    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-certs\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.419772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419628    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccc8b27e-9475-48f3-a202-4bba216e8282-istiod-ca-cert\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.419772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419653    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dc2jw\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-kube-api-access-dc2jw\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.419772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419700    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-credential-socket\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.419772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419721    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-data\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.419772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419749    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-socket\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.420081 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419773    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-envoy\") pod \"ccc8b27e-9475-48f3-a202-4bba216e8282\" (UID: \"ccc8b27e-9475-48f3-a202-4bba216e8282\") "
Jul 29 18:23:53.420081 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.420033    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:53.420194 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.420073    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccc8b27e-9475-48f3-a202-4bba216e8282-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:23:53.420194 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.419968    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:53.420348 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.420219    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:53.421297 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.421239    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-data" (OuterVolumeSpecName: "istio-data") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:53.422126 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.422102    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-token" (OuterVolumeSpecName: "istio-token") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:23:53.422213 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.422190    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-kube-api-access-dc2jw" (OuterVolumeSpecName: "kube-api-access-dc2jw") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "kube-api-access-dc2jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:23:53.422725 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.422705    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:23:53.422828 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.422767    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ccc8b27e-9475-48f3-a202-4bba216e8282" (UID: "ccc8b27e-9475-48f3-a202-4bba216e8282"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:23:53.520957 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.520922    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.520957 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.520951    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccc8b27e-9475-48f3-a202-4bba216e8282-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.520957 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.520962    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc2jw\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-kube-api-access-dc2jw\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.521198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.520971    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.521198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.520981    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.521198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.520989    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.521198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.520997    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.521198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.521004    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.521198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.521011    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccc8b27e-9475-48f3-a202-4bba216e8282-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:23:53.569998 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.569965    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:53.711568 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:53.711539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"]
Jul 29 18:23:53.713191 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:23:53.713167    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c9f051_3d50_4345_93e5_71333abcd4ca.slice/crio-3e0dcb0571241932b3c0b3b3f67348816e16f0dcb676dd08fbcef26b2ff535e8 WatchSource:0}: Error finding container 3e0dcb0571241932b3c0b3b3f67348816e16f0dcb676dd08fbcef26b2ff535e8: Status 404 returned error can't find the container with id 3e0dcb0571241932b3c0b3b3f67348816e16f0dcb676dd08fbcef26b2ff535e8
Jul 29 18:23:54.041766 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.041736    2560 generic.go:356] "Generic (PLEG): container finished" podID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerID="36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618" exitCode=0
Jul 29 18:23:54.041993 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.041788    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"
Jul 29 18:23:54.041993 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.041807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" event={"ID":"ccc8b27e-9475-48f3-a202-4bba216e8282","Type":"ContainerDied","Data":"36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618"}
Jul 29 18:23:54.041993 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.041862    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4" event={"ID":"ccc8b27e-9475-48f3-a202-4bba216e8282","Type":"ContainerDied","Data":"9dfda2f0bbdf5e4e4eb1705decfd755fcbc6d080b54bae56c974ad1f27ff69c3"}
Jul 29 18:23:54.041993 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.041889    2560 scope.go:117] "RemoveContainer" containerID="36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618"
Jul 29 18:23:54.043236 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.043208    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8" event={"ID":"70c9f051-3d50-4345-93e5-71333abcd4ca","Type":"ContainerStarted","Data":"3e0dcb0571241932b3c0b3b3f67348816e16f0dcb676dd08fbcef26b2ff535e8"}
Jul 29 18:23:54.052091 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.052077    2560 scope.go:117] "RemoveContainer" containerID="92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652"
Jul 29 18:23:54.059926 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.059907    2560 scope.go:117] "RemoveContainer" containerID="36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618"
Jul 29 18:23:54.060197 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:54.060175    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618\": container with ID starting with 36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618 not found: ID does not exist" containerID="36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618"
Jul 29 18:23:54.060314 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.060204    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618"} err="failed to get container status \"36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618\": rpc error: code = NotFound desc = could not find container \"36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618\": container with ID starting with 36cd4de8af64eba7c02a6c576f8eb38fb728285dbb0023285ebcabe13c6a9618 not found: ID does not exist"
Jul 29 18:23:54.060314 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.060227    2560 scope.go:117] "RemoveContainer" containerID="92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652"
Jul 29 18:23:54.060589 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:23:54.060554    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652\": container with ID starting with 92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652 not found: ID does not exist" containerID="92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652"
Jul 29 18:23:54.060713 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.060598    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652"} err="failed to get container status \"92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652\": rpc error: code = NotFound desc = could not find container \"92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652\": container with ID starting with 92317436de45ee628b41064175f20120b6b221d4e9e06995ede22956607ac652 not found: ID does not exist"
Jul 29 18:23:54.066391 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.066371    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"]
Jul 29 18:23:54.070820 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.070799    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-697aa257-openshift-default-56cd9b9fc6-2q9f4"]
Jul 29 18:23:54.282400 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:54.282366    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" path="/var/lib/kubelet/pods/ccc8b27e-9475-48f3-a202-4bba216e8282/volumes"
Jul 29 18:23:55.050323 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:55.050292    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8" event={"ID":"70c9f051-3d50-4345-93e5-71333abcd4ca","Type":"ContainerStarted","Data":"3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736"}
Jul 29 18:23:55.050714 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:55.050415    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:23:55.065859 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:23:55.065809    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8" podStartSLOduration=2.683339507 podStartE2EDuration="3.065792002s" totalImagesPullingTime="382.452495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:23:55.065106936 +0000 UTC m=+1825.307068000" watchObservedRunningTime="2026-07-29 18:23:55.065792002 +0000 UTC m=+1825.307753039"
Jul 29 18:24:01.060163 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:01.060132    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:24:16.365489 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365453    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"]
Jul 29 18:24:16.365871 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365852    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy"
Jul 29 18:24:16.365871 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365864    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be00d64-5165-4de2-8a44-ed88c7aece7d" containerName="istio-proxy"
Jul 29 18:24:16.365950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365880    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy"
Jul 29 18:24:16.365950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365886    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy"
Jul 29 18:24:16.365950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365892    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy"
Jul 29 18:24:16.365950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365900    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy"
Jul 29 18:24:16.366062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365987    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy"
Jul 29 18:24:16.366062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.365995    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccc8b27e-9475-48f3-a202-4bba216e8282" containerName="istio-proxy"
Jul 29 18:24:16.369208 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.369192    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"
Jul 29 18:24:16.371027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.371008    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:24:16.375848 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.375399    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"]
Jul 29 18:24:16.447027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.446989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cbbf5c-4d58-4b06-a8de-0e2b9b228069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0eed4609-7t62b\" (UID: \"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"
Jul 29 18:24:16.547999 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.547962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cbbf5c-4d58-4b06-a8de-0e2b9b228069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0eed4609-7t62b\" (UID: \"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"
Jul 29 18:24:16.548678 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.548654    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cbbf5c-4d58-4b06-a8de-0e2b9b228069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0eed4609-7t62b\" (UID: \"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"
Jul 29 18:24:16.681747 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.681667    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"
Jul 29 18:24:16.817875 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:16.817833    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"]
Jul 29 18:24:16.822667 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:24:16.822641    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cbbf5c_4d58_4b06_a8de_0e2b9b228069.slice/crio-48fdc092fc3d79acc0c1a3bda43aed192e55cc80c50c11a0d4cc1fbbe4ed2b52 WatchSource:0}: Error finding container 48fdc092fc3d79acc0c1a3bda43aed192e55cc80c50c11a0d4cc1fbbe4ed2b52: Status 404 returned error can't find the container with id 48fdc092fc3d79acc0c1a3bda43aed192e55cc80c50c11a0d4cc1fbbe4ed2b52
Jul 29 18:24:17.165393 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:17.165352    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b" event={"ID":"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069","Type":"ContainerStarted","Data":"6f0bdb20fcaf34725fbc7a3405e593e605e5ab2538709831a95c52af79c1fa6b"}
Jul 29 18:24:17.165600 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:17.165413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b" event={"ID":"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069","Type":"ContainerStarted","Data":"48fdc092fc3d79acc0c1a3bda43aed192e55cc80c50c11a0d4cc1fbbe4ed2b52"}
Jul 29 18:24:17.181450 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:17.181404    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b" podStartSLOduration=1.18138774 podStartE2EDuration="1.18138774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:24:17.179586287 +0000 UTC m=+1847.421547327" watchObservedRunningTime="2026-07-29 18:24:17.18138774 +0000 UTC m=+1847.423348777"
Jul 29 18:24:18.171195 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:18.171157    2560 generic.go:356] "Generic (PLEG): container finished" podID="f4cbbf5c-4d58-4b06-a8de-0e2b9b228069" containerID="6f0bdb20fcaf34725fbc7a3405e593e605e5ab2538709831a95c52af79c1fa6b" exitCode=0
Jul 29 18:24:18.171585 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:18.171229    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b" event={"ID":"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069","Type":"ContainerDied","Data":"6f0bdb20fcaf34725fbc7a3405e593e605e5ab2538709831a95c52af79c1fa6b"}
Jul 29 18:24:19.310513 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:19.310489    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"
Jul 29 18:24:19.476335 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:19.476230    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cbbf5c-4d58-4b06-a8de-0e2b9b228069-maas-api-service-ca\") pod \"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069\" (UID: \"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069\") "
Jul 29 18:24:19.476636 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:19.476611    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4cbbf5c-4d58-4b06-a8de-0e2b9b228069-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4cbbf5c-4d58-4b06-a8de-0e2b9b228069" (UID: "f4cbbf5c-4d58-4b06-a8de-0e2b9b228069"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:24:19.577657 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:19.577626    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4cbbf5c-4d58-4b06-a8de-0e2b9b228069-maas-api-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:24:20.182774 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:20.182741    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"
Jul 29 18:24:20.182949 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:20.182737    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b" event={"ID":"f4cbbf5c-4d58-4b06-a8de-0e2b9b228069","Type":"ContainerDied","Data":"48fdc092fc3d79acc0c1a3bda43aed192e55cc80c50c11a0d4cc1fbbe4ed2b52"}
Jul 29 18:24:20.182949 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:20.182884    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48fdc092fc3d79acc0c1a3bda43aed192e55cc80c50c11a0d4cc1fbbe4ed2b52"
Jul 29 18:24:23.173560 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.173524    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"]
Jul 29 18:24:23.174124 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.173780    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8" podUID="70c9f051-3d50-4345-93e5-71333abcd4ca" containerName="maas-api" containerID="cri-o://3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736" gracePeriod=30
Jul 29 18:24:23.370595 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.370557    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"]
Jul 29 18:24:23.373475 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.373448    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0eed4609-7t62b"]
Jul 29 18:24:23.426054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.425986    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:24:23.615307 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.615252    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtr7n\" (UniqueName: \"kubernetes.io/projected/70c9f051-3d50-4345-93e5-71333abcd4ca-kube-api-access-wtr7n\") pod \"70c9f051-3d50-4345-93e5-71333abcd4ca\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") "
Jul 29 18:24:23.615481 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.615390    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls\") pod \"70c9f051-3d50-4345-93e5-71333abcd4ca\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") "
Jul 29 18:24:23.615529 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.615486    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c9f051-3d50-4345-93e5-71333abcd4ca-openshift-service-ca\") pod \"70c9f051-3d50-4345-93e5-71333abcd4ca\" (UID: \"70c9f051-3d50-4345-93e5-71333abcd4ca\") "
Jul 29 18:24:23.615843 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.615807    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c9f051-3d50-4345-93e5-71333abcd4ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70c9f051-3d50-4345-93e5-71333abcd4ca" (UID: "70c9f051-3d50-4345-93e5-71333abcd4ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:24:23.617523 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.617500    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70c9f051-3d50-4345-93e5-71333abcd4ca" (UID: "70c9f051-3d50-4345-93e5-71333abcd4ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:24:23.617608 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.617583    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c9f051-3d50-4345-93e5-71333abcd4ca-kube-api-access-wtr7n" (OuterVolumeSpecName: "kube-api-access-wtr7n") pod "70c9f051-3d50-4345-93e5-71333abcd4ca" (UID: "70c9f051-3d50-4345-93e5-71333abcd4ca"). InnerVolumeSpecName "kube-api-access-wtr7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:24:23.716290 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.716190    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c9f051-3d50-4345-93e5-71333abcd4ca-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:24:23.716290 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.716221    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c9f051-3d50-4345-93e5-71333abcd4ca-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:24:23.716290 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:23.716231    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtr7n\" (UniqueName: \"kubernetes.io/projected/70c9f051-3d50-4345-93e5-71333abcd4ca-kube-api-access-wtr7n\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:24:24.203959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.203925    2560 generic.go:356] "Generic (PLEG): container finished" podID="70c9f051-3d50-4345-93e5-71333abcd4ca" containerID="3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736" exitCode=0
Jul 29 18:24:24.204406 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.203972    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"
Jul 29 18:24:24.204406 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.204008    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8" event={"ID":"70c9f051-3d50-4345-93e5-71333abcd4ca","Type":"ContainerDied","Data":"3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736"}
Jul 29 18:24:24.204406 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.204066    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8" event={"ID":"70c9f051-3d50-4345-93e5-71333abcd4ca","Type":"ContainerDied","Data":"3e0dcb0571241932b3c0b3b3f67348816e16f0dcb676dd08fbcef26b2ff535e8"}
Jul 29 18:24:24.204406 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.204090    2560 scope.go:117] "RemoveContainer" containerID="3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736"
Jul 29 18:24:24.213299 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.213259    2560 scope.go:117] "RemoveContainer" containerID="3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736"
Jul 29 18:24:24.213570 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:24:24.213551    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736\": container with ID starting with 3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736 not found: ID does not exist" containerID="3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736"
Jul 29 18:24:24.213625 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.213580    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736"} err="failed to get container status \"3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736\": rpc error: code = NotFound desc = could not find container \"3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736\": container with ID starting with 3c9744d5f48924c4d549a970ad24be784b17a407d272d96c06e9c198655be736 not found: ID does not exist"
Jul 29 18:24:24.225478 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.225452    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"]
Jul 29 18:24:24.231128 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.231103    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0eed4609-bccbb58f9-gr6c8"]
Jul 29 18:24:24.281828 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.281799    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c9f051-3d50-4345-93e5-71333abcd4ca" path="/var/lib/kubelet/pods/70c9f051-3d50-4345-93e5-71333abcd4ca/volumes"
Jul 29 18:24:24.282136 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:24:24.282124    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4cbbf5c-4d58-4b06-a8de-0e2b9b228069" path="/var/lib/kubelet/pods/f4cbbf5c-4d58-4b06-a8de-0e2b9b228069/volumes"
Jul 29 18:26:07.262734 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.262703    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"]
Jul 29 18:26:07.263227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.263150    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4cbbf5c-4d58-4b06-a8de-0e2b9b228069" containerName="revoke-keys"
Jul 29 18:26:07.263227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.263168    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4cbbf5c-4d58-4b06-a8de-0e2b9b228069" containerName="revoke-keys"
Jul 29 18:26:07.263227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.263181    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c9f051-3d50-4345-93e5-71333abcd4ca" containerName="maas-api"
Jul 29 18:26:07.263227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.263190    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c9f051-3d50-4345-93e5-71333abcd4ca" containerName="maas-api"
Jul 29 18:26:07.263437 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.263261    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c9f051-3d50-4345-93e5-71333abcd4ca" containerName="maas-api"
Jul 29 18:26:07.263437 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.263294    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4cbbf5c-4d58-4b06-a8de-0e2b9b228069" containerName="revoke-keys"
Jul 29 18:26:07.266448 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.266431    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.268495 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.268473    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:26:07.268669 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.268651    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0cdae181-openshift-default-dockercfg-9dspx\""
Jul 29 18:26:07.296781 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.296743    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"]
Jul 29 18:26:07.404705 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-socket\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.404705 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-data\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.404959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-token\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.404959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-certs\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.404959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-podinfo\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.404959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404800    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5rp2\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-kube-api-access-h5rp2\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.404959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404854    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cca9395-7377-40da-9a5d-469d75c70fa6-istiod-ca-cert\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.404959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.404886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-envoy\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.405227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.405008    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-credential-socket\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506151 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506113    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rp2\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-kube-api-access-h5rp2\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506366 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506161    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cca9395-7377-40da-9a5d-469d75c70fa6-istiod-ca-cert\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506366 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506322    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-envoy\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506453 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506417    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-credential-socket\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506489 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506474    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-socket\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506523 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506501    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-data\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506569 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506529    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-token\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506569 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506557    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-certs\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506670 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-podinfo\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506818 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-credential-socket\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.506981 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506956    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cca9395-7377-40da-9a5d-469d75c70fa6-istiod-ca-cert\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.507087 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.507007    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-certs\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.507087 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506972    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-data\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.507087 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.506846    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-socket\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.508854 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.508836    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-envoy\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.509093 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.509074    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-podinfo\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.515228 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.515174    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-token\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.515374 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.515358    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5rp2\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-kube-api-access-h5rp2\") pod \"e2e-mt-0cdae181-openshift-default-855d648fb-nchb5\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") " pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.577287 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.577231    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:07.718077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.718011    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"]
Jul 29 18:26:07.720672 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:26:07.720636    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cca9395_7377_40da_9a5d_469d75c70fa6.slice/crio-6ffd1c59cb4a0d800a7d4b8603af402da7707961afc2db9cbe4a22ae26007934 WatchSource:0}: Error finding container 6ffd1c59cb4a0d800a7d4b8603af402da7707961afc2db9cbe4a22ae26007934: Status 404 returned error can't find the container with id 6ffd1c59cb4a0d800a7d4b8603af402da7707961afc2db9cbe4a22ae26007934
Jul 29 18:26:07.723007 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.722969    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:26:07.723125 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.723052    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:26:07.723125 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.723102    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:26:07.734215 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:07.734193    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" event={"ID":"2cca9395-7377-40da-9a5d-469d75c70fa6","Type":"ContainerStarted","Data":"6ffd1c59cb4a0d800a7d4b8603af402da7707961afc2db9cbe4a22ae26007934"}
Jul 29 18:26:08.740510 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:08.740471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" event={"ID":"2cca9395-7377-40da-9a5d-469d75c70fa6","Type":"ContainerStarted","Data":"80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5"}
Jul 29 18:26:08.760938 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:08.760887    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podStartSLOduration=1.7608712039999999 podStartE2EDuration="1.760871204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:08.759025175 +0000 UTC m=+1959.000986224" watchObservedRunningTime="2026-07-29 18:26:08.760871204 +0000 UTC m=+1959.002832241"
Jul 29 18:26:09.578315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:09.578253    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:09.582967 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:09.582941    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:09.745436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:09.745411    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:09.746290 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:09.746248    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:11.870592 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:11.870554    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"]
Jul 29 18:26:11.874801 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:11.874769    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:11.876996 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:11.876970    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0cdae181-serving-cert\""
Jul 29 18:26:11.887215 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:11.887188    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"]
Jul 29 18:26:11.957077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:11.957039    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c8d346-dd74-480d-b467-222c0ee48f30-openshift-service-ca\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:11.957251 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:11.957164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr7qb\" (UniqueName: \"kubernetes.io/projected/28c8d346-dd74-480d-b467-222c0ee48f30-kube-api-access-zr7qb\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:11.957340 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:11.957245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28c8d346-dd74-480d-b467-222c0ee48f30-maas-api-tls\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.058731 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.058698    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr7qb\" (UniqueName: \"kubernetes.io/projected/28c8d346-dd74-480d-b467-222c0ee48f30-kube-api-access-zr7qb\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.058902 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.058756    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28c8d346-dd74-480d-b467-222c0ee48f30-maas-api-tls\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.058902 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.058818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c8d346-dd74-480d-b467-222c0ee48f30-openshift-service-ca\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.059537 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.059518    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c8d346-dd74-480d-b467-222c0ee48f30-openshift-service-ca\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.061308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.061288    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28c8d346-dd74-480d-b467-222c0ee48f30-maas-api-tls\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.070985 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.070960    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr7qb\" (UniqueName: \"kubernetes.io/projected/28c8d346-dd74-480d-b467-222c0ee48f30-kube-api-access-zr7qb\") pod \"maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.188923 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.188826    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:12.355655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.355628    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"]
Jul 29 18:26:12.357113 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:26:12.357088    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c8d346_dd74_480d_b467_222c0ee48f30.slice/crio-6d55425aab295d625bda58d62179253607d39abc8767fb6a7d40a840ba5dc5c2 WatchSource:0}: Error finding container 6d55425aab295d625bda58d62179253607d39abc8767fb6a7d40a840ba5dc5c2: Status 404 returned error can't find the container with id 6d55425aab295d625bda58d62179253607d39abc8767fb6a7d40a840ba5dc5c2
Jul 29 18:26:12.761902 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:12.761867    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4" event={"ID":"28c8d346-dd74-480d-b467-222c0ee48f30","Type":"ContainerStarted","Data":"6d55425aab295d625bda58d62179253607d39abc8767fb6a7d40a840ba5dc5c2"}
Jul 29 18:26:13.768440 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:13.768399    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4" event={"ID":"28c8d346-dd74-480d-b467-222c0ee48f30","Type":"ContainerStarted","Data":"ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df"}
Jul 29 18:26:13.768848 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:13.768469    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:13.791077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:13.791021    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4" podStartSLOduration=2.361994171 podStartE2EDuration="2.791004911s" totalImagesPullingTime="429.01074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:13.788313293 +0000 UTC m=+1964.030274331" watchObservedRunningTime="2026-07-29 18:26:13.791004911 +0000 UTC m=+1964.032966003"
Jul 29 18:26:19.779181 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:19.779146    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:24.746780 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.746742    2560 patch_prober.go:28] interesting pod/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:26:24.747124 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.746822    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:26:24.833671 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.833635    2560 generic.go:356] "Generic (PLEG): container finished" podID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerID="80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5" exitCode=0
Jul 29 18:26:24.833846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.833708    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" event={"ID":"2cca9395-7377-40da-9a5d-469d75c70fa6","Type":"ContainerDied","Data":"80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5"}
Jul 29 18:26:24.834117 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.834103    2560 scope.go:117] "RemoveContainer" containerID="80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5"
Jul 29 18:26:24.835132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.835106    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:26:24.835228 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.835189    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:26:24.835313 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:24.835237    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:26:25.842757 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:25.842723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" event={"ID":"2cca9395-7377-40da-9a5d-469d75c70fa6","Type":"ContainerStarted","Data":"4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed"}
Jul 29 18:26:26.577608 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:26.577570    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:26.577792 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:26.577753    2560 patch_prober.go:28] interesting pod/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:26:26.577838 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:26.577798    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:26:27.578424 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:27.578382    2560 patch_prober.go:28] interesting pod/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:26:27.578890 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:27.578445    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:26:28.578414 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:28.578373    2560 patch_prober.go:28] interesting pod/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:26:28.578596 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:28.578446    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:26:29.577759 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:29.577717    2560 patch_prober.go:28] interesting pod/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:26:29.577942 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:29.577776    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:26:30.581776 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:30.581744    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:30.582190 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:30.582147    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:30.582970 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:30.582944    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:26:53.497004 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.496970    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"]
Jul 29 18:26:53.497628 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.497186    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4" podUID="28c8d346-dd74-480d-b467-222c0ee48f30" containerName="maas-api" containerID="cri-o://ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df" gracePeriod=30
Jul 29 18:26:53.747110 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.747047    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:53.875011 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.874972    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28c8d346-dd74-480d-b467-222c0ee48f30-maas-api-tls\") pod \"28c8d346-dd74-480d-b467-222c0ee48f30\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") "
Jul 29 18:26:53.875183 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.875064    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr7qb\" (UniqueName: \"kubernetes.io/projected/28c8d346-dd74-480d-b467-222c0ee48f30-kube-api-access-zr7qb\") pod \"28c8d346-dd74-480d-b467-222c0ee48f30\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") "
Jul 29 18:26:53.875183 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.875096    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c8d346-dd74-480d-b467-222c0ee48f30-openshift-service-ca\") pod \"28c8d346-dd74-480d-b467-222c0ee48f30\" (UID: \"28c8d346-dd74-480d-b467-222c0ee48f30\") "
Jul 29 18:26:53.875547 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.875521    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c8d346-dd74-480d-b467-222c0ee48f30-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28c8d346-dd74-480d-b467-222c0ee48f30" (UID: "28c8d346-dd74-480d-b467-222c0ee48f30"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:26:53.877318 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.877286    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c8d346-dd74-480d-b467-222c0ee48f30-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28c8d346-dd74-480d-b467-222c0ee48f30" (UID: "28c8d346-dd74-480d-b467-222c0ee48f30"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:26:53.877413 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.877364    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c8d346-dd74-480d-b467-222c0ee48f30-kube-api-access-zr7qb" (OuterVolumeSpecName: "kube-api-access-zr7qb") pod "28c8d346-dd74-480d-b467-222c0ee48f30" (UID: "28c8d346-dd74-480d-b467-222c0ee48f30"). InnerVolumeSpecName "kube-api-access-zr7qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:26:53.975898 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.975849    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28c8d346-dd74-480d-b467-222c0ee48f30-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:26:53.975898 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.975892    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr7qb\" (UniqueName: \"kubernetes.io/projected/28c8d346-dd74-480d-b467-222c0ee48f30-kube-api-access-zr7qb\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:26:53.975898 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.975907    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c8d346-dd74-480d-b467-222c0ee48f30-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:26:53.991436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.991398    2560 generic.go:356] "Generic (PLEG): container finished" podID="28c8d346-dd74-480d-b467-222c0ee48f30" containerID="ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df" exitCode=0
Jul 29 18:26:53.991614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.991451    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"
Jul 29 18:26:53.991614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.991495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4" event={"ID":"28c8d346-dd74-480d-b467-222c0ee48f30","Type":"ContainerDied","Data":"ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df"}
Jul 29 18:26:53.991614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.991563    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4" event={"ID":"28c8d346-dd74-480d-b467-222c0ee48f30","Type":"ContainerDied","Data":"6d55425aab295d625bda58d62179253607d39abc8767fb6a7d40a840ba5dc5c2"}
Jul 29 18:26:53.991614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:53.991595    2560 scope.go:117] "RemoveContainer" containerID="ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df"
Jul 29 18:26:54.001156 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:54.001139    2560 scope.go:117] "RemoveContainer" containerID="ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df"
Jul 29 18:26:54.001440 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:26:54.001418    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df\": container with ID starting with ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df not found: ID does not exist" containerID="ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df"
Jul 29 18:26:54.001503 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:54.001452    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df"} err="failed to get container status \"ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df\": rpc error: code = NotFound desc = could not find container \"ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df\": container with ID starting with ab450035f8117a94bdb0c1cf00c56d6ef2c16fbd3acd619ce1ae71f70ba109df not found: ID does not exist"
Jul 29 18:26:54.018449 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:54.018425    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"]
Jul 29 18:26:54.022652 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:54.022632    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cdae181-7bbc8f6df7-759k4"]
Jul 29 18:26:54.283338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:26:54.283239    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c8d346-dd74-480d-b467-222c0ee48f30" path="/var/lib/kubelet/pods/28c8d346-dd74-480d-b467-222c0ee48f30/volumes"
Jul 29 18:27:00.101844 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:00.101812    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"]
Jul 29 18:27:00.102312 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:00.102066    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy" containerID="cri-o://4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed" gracePeriod=30
Jul 29 18:27:00.584723 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:00.584682    2560 patch_prober.go:28] interesting pod/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:27:00.584918 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:00.584753    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:27:05.350766 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.350738    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:27:05.381775 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.381693    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5rp2\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-kube-api-access-h5rp2\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.381775 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.381731    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-token\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.382008 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.381779    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-data\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.382008 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.381798    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-podinfo\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.382008 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.381828    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-envoy\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.382290    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-credential-socket\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.382360    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cca9395-7377-40da-9a5d-469d75c70fa6-istiod-ca-cert\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.382467    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-socket\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.382618    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-certs\") pod \"2cca9395-7377-40da-9a5d-469d75c70fa6\" (UID: \"2cca9395-7377-40da-9a5d-469d75c70fa6\") "
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.382827    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.382914    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-data" (OuterVolumeSpecName: "istio-data") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383032    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383213    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383340    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cca9395-7377-40da-9a5d-469d75c70fa6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383632    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383653    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383668    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cca9395-7377-40da-9a5d-469d75c70fa6-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383682    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.384062 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.383695    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.385244 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.385222    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-kube-api-access-h5rp2" (OuterVolumeSpecName: "kube-api-access-h5rp2") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "kube-api-access-h5rp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:27:05.385599 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.385572    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:27:05.385703 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.385679    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-token" (OuterVolumeSpecName: "istio-token") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:27:05.386396 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.386375    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2cca9395-7377-40da-9a5d-469d75c70fa6" (UID: "2cca9395-7377-40da-9a5d-469d75c70fa6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:05.484559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.484523    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5rp2\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-kube-api-access-h5rp2\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.484559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.484554    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.484559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.484564    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:05.484801 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:05.484573    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cca9395-7377-40da-9a5d-469d75c70fa6-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:27:06.052727 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.052692    2560 generic.go:356] "Generic (PLEG): container finished" podID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerID="4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed" exitCode=0
Jul 29 18:27:06.052931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.052749    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"
Jul 29 18:27:06.052931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.052776    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" event={"ID":"2cca9395-7377-40da-9a5d-469d75c70fa6","Type":"ContainerDied","Data":"4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed"}
Jul 29 18:27:06.052931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.052831    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5" event={"ID":"2cca9395-7377-40da-9a5d-469d75c70fa6","Type":"ContainerDied","Data":"6ffd1c59cb4a0d800a7d4b8603af402da7707961afc2db9cbe4a22ae26007934"}
Jul 29 18:27:06.052931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.052854    2560 scope.go:117] "RemoveContainer" containerID="4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed"
Jul 29 18:27:06.062259 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.062243    2560 scope.go:117] "RemoveContainer" containerID="80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5"
Jul 29 18:27:06.071069 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.071046    2560 scope.go:117] "RemoveContainer" containerID="4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed"
Jul 29 18:27:06.071362 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:27:06.071344    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed\": container with ID starting with 4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed not found: ID does not exist" containerID="4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed"
Jul 29 18:27:06.071430 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.071374    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed"} err="failed to get container status \"4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed\": rpc error: code = NotFound desc = could not find container \"4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed\": container with ID starting with 4bf519e76c7634fb7d475e342c7686d8ed391eb6786b546f9e5ba2e7ef8410ed not found: ID does not exist"
Jul 29 18:27:06.071430 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.071394    2560 scope.go:117] "RemoveContainer" containerID="80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5"
Jul 29 18:27:06.071708 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:27:06.071672    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5\": container with ID starting with 80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5 not found: ID does not exist" containerID="80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5"
Jul 29 18:27:06.071892 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.071713    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5"} err="failed to get container status \"80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5\": rpc error: code = NotFound desc = could not find container \"80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5\": container with ID starting with 80faf4d6b9783e7cc20fd1b5172313806c7e9eb191e7e2e7df403cbb03a324f5 not found: ID does not exist"
Jul 29 18:27:06.073998 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.073976    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"]
Jul 29 18:27:06.078357 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.078333    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0cdae181-openshift-default-855d648fb-nchb5"]
Jul 29 18:27:06.282066 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:06.282033    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" path="/var/lib/kubelet/pods/2cca9395-7377-40da-9a5d-469d75c70fa6/volumes"
Jul 29 18:27:19.785482 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785446    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"]
Jul 29 18:27:19.785846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785831    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy"
Jul 29 18:27:19.785846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785841    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy"
Jul 29 18:27:19.785921 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785854    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy"
Jul 29 18:27:19.785921 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785859    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy"
Jul 29 18:27:19.785921 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785878    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c8d346-dd74-480d-b467-222c0ee48f30" containerName="maas-api"
Jul 29 18:27:19.785921 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785884    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c8d346-dd74-480d-b467-222c0ee48f30" containerName="maas-api"
Jul 29 18:27:19.786041 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785943    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c8d346-dd74-480d-b467-222c0ee48f30" containerName="maas-api"
Jul 29 18:27:19.786041 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785951    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy"
Jul 29 18:27:19.786041 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.785956    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cca9395-7377-40da-9a5d-469d75c70fa6" containerName="istio-proxy"
Jul 29 18:27:19.790726 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.790704    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.792845 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.792823    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:27:19.792946 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.792875    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9245df36-openshift-default-dockercfg-m9d6k\""
Jul 29 18:27:19.802240 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.802216    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"]
Jul 29 18:27:19.924150 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924107    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istiod-ca-cert\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924342 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-credential-socket\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924342 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924300    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-socket\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924342 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924329    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-data\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924474 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-envoy\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924474 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924377    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-podinfo\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924474 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-token\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924474 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924469    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-449f2\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-kube-api-access-449f2\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:19.924606 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:19.924514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-certs\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025099 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istiod-ca-cert\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025099 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-credential-socket\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025150    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-socket\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025165    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-data\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025188    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-envoy\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025206    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-podinfo\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-token\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025293    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-449f2\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-kube-api-access-449f2\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025334    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-certs\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025787 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025759    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-credential-socket\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025876 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025803    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-socket\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.025876 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-certs\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.026030 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.025887    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-data\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.026202 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.026161    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istiod-ca-cert\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.027675 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.027657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-envoy\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.027902 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.027885    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-podinfo\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.032810 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.032782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-token\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.033257 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.033235    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-449f2\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-kube-api-access-449f2\") pod \"e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") " pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.101628 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.101532    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:20.241443 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.241417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"]
Jul 29 18:27:20.243557 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:27:20.243529    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod227f8d5e_e4b2_4022_8e88_2f122805f0f6.slice/crio-b6de58d44f3ae2ddea519680f99fb02e4703ad640c75a8189505e903e0ae5898 WatchSource:0}: Error finding container b6de58d44f3ae2ddea519680f99fb02e4703ad640c75a8189505e903e0ae5898: Status 404 returned error can't find the container with id b6de58d44f3ae2ddea519680f99fb02e4703ad640c75a8189505e903e0ae5898
Jul 29 18:27:20.245908 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.245868    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:27:20.245979 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.245962    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:27:20.246024 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:20.246008    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:27:21.134168 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:21.134130    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" event={"ID":"227f8d5e-e4b2-4022-8e88-2f122805f0f6","Type":"ContainerStarted","Data":"699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc"}
Jul 29 18:27:21.134618 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:21.134185    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" event={"ID":"227f8d5e-e4b2-4022-8e88-2f122805f0f6","Type":"ContainerStarted","Data":"b6de58d44f3ae2ddea519680f99fb02e4703ad640c75a8189505e903e0ae5898"}
Jul 29 18:27:21.155596 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:21.155540    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podStartSLOduration=2.155523911 podStartE2EDuration="2.155523911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:27:21.154071591 +0000 UTC m=+2031.396032629" watchObservedRunningTime="2026-07-29 18:27:21.155523911 +0000 UTC m=+2031.397484951"
Jul 29 18:27:22.102645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:22.102608    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:22.107461 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:22.107436    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:22.139865 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:22.139839    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:22.140832 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:22.140816    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:24.132593 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.132557    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"]
Jul 29 18:27:24.137052 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.137025    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.139712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.139692    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9245df36-serving-cert\""
Jul 29 18:27:24.146843 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.146818    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"]
Jul 29 18:27:24.265709 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.265671    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt7s6\" (UniqueName: \"kubernetes.io/projected/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-kube-api-access-pt7s6\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.265997 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.265719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.265997 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.265787    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-openshift-service-ca\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.366987 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.366949    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-openshift-service-ca\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.367158 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.367061    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt7s6\" (UniqueName: \"kubernetes.io/projected/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-kube-api-access-pt7s6\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.367158 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.367089    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.367232 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:27:24.367206    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-serving-cert: secret "maas-api-e2e-mt-9245df36-serving-cert" not found
Jul 29 18:27:24.367327 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:27:24.367315    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls podName:1cff37ee-0b39-4b4a-b15f-5eabfb4694c7 nodeName:}" failed. No retries permitted until 2026-07-29 18:27:24.867262046 +0000 UTC m=+2035.109223063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls") pod "maas-api-e2e-mt-9245df36-79b7f847c7-49nhl" (UID: "1cff37ee-0b39-4b4a-b15f-5eabfb4694c7") : secret "maas-api-e2e-mt-9245df36-serving-cert" not found
Jul 29 18:27:24.367560 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.367541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-openshift-service-ca\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.375352 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.375322    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt7s6\" (UniqueName: \"kubernetes.io/projected/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-kube-api-access-pt7s6\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.871670 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.871633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:24.874490 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:24.874459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls\") pod \"maas-api-e2e-mt-9245df36-79b7f847c7-49nhl\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:25.052914 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:25.052872    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:25.191920 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:25.191894    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"]
Jul 29 18:27:25.194904 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:27:25.194877    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cff37ee_0b39_4b4a_b15f_5eabfb4694c7.slice/crio-81dac2bb499652baec2f65734d895d6f5691746afbfe74d96196ae606a607e19 WatchSource:0}: Error finding container 81dac2bb499652baec2f65734d895d6f5691746afbfe74d96196ae606a607e19: Status 404 returned error can't find the container with id 81dac2bb499652baec2f65734d895d6f5691746afbfe74d96196ae606a607e19
Jul 29 18:27:25.196953 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:25.196934    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:27:26.165731 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:26.165682    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl" event={"ID":"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7","Type":"ContainerStarted","Data":"e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97"}
Jul 29 18:27:26.165936 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:26.165748    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl" event={"ID":"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7","Type":"ContainerStarted","Data":"81dac2bb499652baec2f65734d895d6f5691746afbfe74d96196ae606a607e19"}
Jul 29 18:27:26.165936 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:26.165874    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:26.182304 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:26.182241    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl" podStartSLOduration=1.8371037220000002 podStartE2EDuration="2.182224492s" totalImagesPullingTime="345.12077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:27:26.180038196 +0000 UTC m=+2036.421999235" watchObservedRunningTime="2026-07-29 18:27:26.182224492 +0000 UTC m=+2036.424185530"
Jul 29 18:27:32.179077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:32.179046    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:27:37.939735 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:37.939529    2560 patch_prober.go:28] interesting pod/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": read tcp 10.134.0.2:37836->10.134.0.51:15021: read: connection reset by peer" start-of-body=
Jul 29 18:27:37.939735 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:37.939610    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": read tcp 10.134.0.2:37836->10.134.0.51:15021: read: connection reset by peer"
Jul 29 18:27:38.235567 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:38.235459    2560 generic.go:356] "Generic (PLEG): container finished" podID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerID="699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc" exitCode=0
Jul 29 18:27:38.235567 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:38.235535    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" event={"ID":"227f8d5e-e4b2-4022-8e88-2f122805f0f6","Type":"ContainerDied","Data":"699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc"}
Jul 29 18:27:38.235962 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:38.235946    2560 scope.go:117] "RemoveContainer" containerID="699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc"
Jul 29 18:27:38.237080 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:38.237053    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:27:38.237163 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:38.237143    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:27:38.237200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:38.237188    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:27:39.243999 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:39.243970    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" event={"ID":"227f8d5e-e4b2-4022-8e88-2f122805f0f6","Type":"ContainerStarted","Data":"914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a"}
Jul 29 18:27:40.102770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:40.102722    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:40.103000 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:40.102975    2560 patch_prober.go:28] interesting pod/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:27:40.103082 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:40.103048    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:27:41.102065 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:41.102017    2560 patch_prober.go:28] interesting pod/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:27:41.102565 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:41.102083    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:27:42.102621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:42.102585    2560 patch_prober.go:28] interesting pod/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:27:42.102990 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:42.102640    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:27:43.102564 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:43.102527    2560 patch_prober.go:28] interesting pod/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:27:43.102766 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:43.102588    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:27:44.105969 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:44.105932    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:44.106432 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:44.106198    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:27:44.106811 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:27:44.106796    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:28:30.323541 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:30.323511    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:28:30.327016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:30.326990    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:28:30.410380 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:30.410347    2560 scope.go:117] "RemoveContainer" containerID="04dd0c79720f151656ac82c90f42e045bc492c075f58ad220f0e66937906d04c"
Jul 29 18:28:43.034130 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.034080    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"]
Jul 29 18:28:43.034630 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.034322    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl" podUID="1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" containerName="maas-api" containerID="cri-o://e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97" gracePeriod=30
Jul 29 18:28:43.296031 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.295962    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:28:43.439003 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.438969    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls\") pod \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") "
Jul 29 18:28:43.439186 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.439080    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt7s6\" (UniqueName: \"kubernetes.io/projected/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-kube-api-access-pt7s6\") pod \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") "
Jul 29 18:28:43.439186 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.439100    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-openshift-service-ca\") pod \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\" (UID: \"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7\") "
Jul 29 18:28:43.439628 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.439592    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" (UID: "1cff37ee-0b39-4b4a-b15f-5eabfb4694c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:28:43.441243 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.441208    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" (UID: "1cff37ee-0b39-4b4a-b15f-5eabfb4694c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:28:43.441377 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.441331    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-kube-api-access-pt7s6" (OuterVolumeSpecName: "kube-api-access-pt7s6") pod "1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" (UID: "1cff37ee-0b39-4b4a-b15f-5eabfb4694c7"). InnerVolumeSpecName "kube-api-access-pt7s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:28:43.540033 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.540003    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:28:43.540033 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.540030    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt7s6\" (UniqueName: \"kubernetes.io/projected/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-kube-api-access-pt7s6\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:28:43.540033 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.540039    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:28:43.590578 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.590496    2560 generic.go:356] "Generic (PLEG): container finished" podID="1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" containerID="e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97" exitCode=0
Jul 29 18:28:43.590578 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.590542    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"
Jul 29 18:28:43.590770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.590581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl" event={"ID":"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7","Type":"ContainerDied","Data":"e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97"}
Jul 29 18:28:43.590770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.590637    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl" event={"ID":"1cff37ee-0b39-4b4a-b15f-5eabfb4694c7","Type":"ContainerDied","Data":"81dac2bb499652baec2f65734d895d6f5691746afbfe74d96196ae606a607e19"}
Jul 29 18:28:43.590770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.590659    2560 scope.go:117] "RemoveContainer" containerID="e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97"
Jul 29 18:28:43.600898 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.600884    2560 scope.go:117] "RemoveContainer" containerID="e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97"
Jul 29 18:28:43.601166 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:28:43.601147    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97\": container with ID starting with e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97 not found: ID does not exist" containerID="e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97"
Jul 29 18:28:43.601238 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.601174    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97"} err="failed to get container status \"e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97\": rpc error: code = NotFound desc = could not find container \"e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97\": container with ID starting with e7e4faf9c19f83bd9e9c01fde10d9c66f11241a4eab8e20b1c9c77d6e1823b97 not found: ID does not exist"
Jul 29 18:28:43.615523 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.615495    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"]
Jul 29 18:28:43.619190 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:43.619170    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9245df36-79b7f847c7-49nhl"]
Jul 29 18:28:44.282335 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:28:44.282306    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" path="/var/lib/kubelet/pods/1cff37ee-0b39-4b4a-b15f-5eabfb4694c7/volumes"
Jul 29 18:29:28.170374 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:28.170259    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"]
Jul 29 18:29:28.170881 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:28.170588    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy" containerID="cri-o://914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a" gracePeriod=30
Jul 29 18:29:29.108105 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:29.108068    2560 patch_prober.go:28] interesting pod/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:29:29.108296 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:29.108140    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:29:30.430607 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:30.430567    2560 scope.go:117] "RemoveContainer" containerID="e5c497d923ed55d6ccc9d2ae314f330ba207032f99dcc0e729c4fb76ccb5dd38"
Jul 29 18:29:30.439077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:30.439056    2560 scope.go:117] "RemoveContainer" containerID="b065b8c82c63ed5b05634c0c70b6482fbce5f972052b2c0cad2a098a6c26e24b"
Jul 29 18:29:33.433109 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.433041    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:29:33.524210 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524175    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-token\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524223    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-envoy\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524254    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-credential-socket\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524300    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-certs\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524340    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-socket\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524410 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524368    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-podinfo\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524453    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-449f2\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-kube-api-access-449f2\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524484    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-data\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524523    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istiod-ca-cert\") pod \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\" (UID: \"227f8d5e-e4b2-4022-8e88-2f122805f0f6\") "
Jul 29 18:29:33.524712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524593    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:29:33.524712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524623    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:29:33.524961 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524837    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:29:33.524961 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524929    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.524961 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524943    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:29:33.524961 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524949    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.525170 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.524979    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.525330 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.525309    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:29:33.527141 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.527117    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:29:33.527141 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.527120    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:29:33.527565 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.527533    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:29:33.527646 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.527574    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-kube-api-access-449f2" (OuterVolumeSpecName: "kube-api-access-449f2") pod "227f8d5e-e4b2-4022-8e88-2f122805f0f6" (UID: "227f8d5e-e4b2-4022-8e88-2f122805f0f6"). InnerVolumeSpecName "kube-api-access-449f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:29:33.625579 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.625543    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.625579 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.625572    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.625579 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.625581    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.625579 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.625590    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.625846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.625599    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-449f2\" (UniqueName: \"kubernetes.io/projected/227f8d5e-e4b2-4022-8e88-2f122805f0f6-kube-api-access-449f2\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.625846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.625609    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227f8d5e-e4b2-4022-8e88-2f122805f0f6-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:29:33.847351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.847317    2560 generic.go:356] "Generic (PLEG): container finished" podID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerID="914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a" exitCode=0
Jul 29 18:29:33.847521 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.847374    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"
Jul 29 18:29:33.847521 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.847390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" event={"ID":"227f8d5e-e4b2-4022-8e88-2f122805f0f6","Type":"ContainerDied","Data":"914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a"}
Jul 29 18:29:33.847521 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.847458    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh" event={"ID":"227f8d5e-e4b2-4022-8e88-2f122805f0f6","Type":"ContainerDied","Data":"b6de58d44f3ae2ddea519680f99fb02e4703ad640c75a8189505e903e0ae5898"}
Jul 29 18:29:33.847521 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.847479    2560 scope.go:117] "RemoveContainer" containerID="914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a"
Jul 29 18:29:33.856820 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.856801    2560 scope.go:117] "RemoveContainer" containerID="699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc"
Jul 29 18:29:33.864981 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.864961    2560 scope.go:117] "RemoveContainer" containerID="914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a"
Jul 29 18:29:33.865245 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:29:33.865224    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a\": container with ID starting with 914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a not found: ID does not exist" containerID="914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a"
Jul 29 18:29:33.865334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.865253    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a"} err="failed to get container status \"914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a\": rpc error: code = NotFound desc = could not find container \"914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a\": container with ID starting with 914c53eb83ee6bd8f53985c5e5468b4c8480a1ab7323f515b1c1efede138fd7a not found: ID does not exist"
Jul 29 18:29:33.865334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.865298    2560 scope.go:117] "RemoveContainer" containerID="699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc"
Jul 29 18:29:33.865557 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:29:33.865539    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc\": container with ID starting with 699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc not found: ID does not exist" containerID="699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc"
Jul 29 18:29:33.865611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.865564    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc"} err="failed to get container status \"699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc\": rpc error: code = NotFound desc = could not find container \"699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc\": container with ID starting with 699baa3a8f037bc6e4d527b06aa90c708846cf506779bd9165f10d413b9d62fc not found: ID does not exist"
Jul 29 18:29:33.875888 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.875860    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"]
Jul 29 18:29:33.898441 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:33.898407    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9245df36-openshift-default-6965f4b544-jzlxh"]
Jul 29 18:29:34.281555 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:29:34.281522    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" path="/var/lib/kubelet/pods/227f8d5e-e4b2-4022-8e88-2f122805f0f6/volumes"
Jul 29 18:30:00.138833 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.138800    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj"]
Jul 29 18:30:00.139332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.139176    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy"
Jul 29 18:30:00.139332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.139186    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy"
Jul 29 18:30:00.139332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.139196    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" containerName="maas-api"
Jul 29 18:30:00.139332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.139202    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" containerName="maas-api"
Jul 29 18:30:00.139332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.139318    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cff37ee-0b39-4b4a-b15f-5eabfb4694c7" containerName="maas-api"
Jul 29 18:30:00.139332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.139330    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy"
Jul 29 18:30:00.139533 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.139339    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy"
Jul 29 18:30:00.142382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.142365    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj"
Jul 29 18:30:00.144688 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.144668    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:30:00.152296 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.152258    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj"]
Jul 29 18:30:00.152467 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.152447    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj"
Jul 29 18:30:00.286104 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.286078    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj"]
Jul 29 18:30:00.287757 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:30:00.287731    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a9b25f_8529_46b8_8cef_d0a0644e3e70.slice/crio-4d8353f3fd232c5c409a40c7ea93cd0efa6c2d7f518145c854b592f1e16b08ed WatchSource:0}: Error finding container 4d8353f3fd232c5c409a40c7ea93cd0efa6c2d7f518145c854b592f1e16b08ed: Status 404 returned error can't find the container with id 4d8353f3fd232c5c409a40c7ea93cd0efa6c2d7f518145c854b592f1e16b08ed
Jul 29 18:30:00.981808 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.981703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj" event={"ID":"94a9b25f-8529-46b8-8cef-d0a0644e3e70","Type":"ContainerStarted","Data":"fd788b5ea4d049faa5f6d184dcf520d83ac942368c726ef7d32f516a3fc146c0"}
Jul 29 18:30:00.981808 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.981795    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj" event={"ID":"94a9b25f-8529-46b8-8cef-d0a0644e3e70","Type":"ContainerStarted","Data":"4d8353f3fd232c5c409a40c7ea93cd0efa6c2d7f518145c854b592f1e16b08ed"}
Jul 29 18:30:00.997295 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:00.997228    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj" podStartSLOduration=0.997211804 podStartE2EDuration="997.211804ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:30:00.994569586 +0000 UTC m=+2191.236530628" watchObservedRunningTime="2026-07-29 18:30:00.997211804 +0000 UTC m=+2191.239172841"
Jul 29 18:30:01.988021 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:01.987984    2560 generic.go:356] "Generic (PLEG): container finished" podID="94a9b25f-8529-46b8-8cef-d0a0644e3e70" containerID="fd788b5ea4d049faa5f6d184dcf520d83ac942368c726ef7d32f516a3fc146c0" exitCode=0
Jul 29 18:30:01.988420 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:01.988058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj" event={"ID":"94a9b25f-8529-46b8-8cef-d0a0644e3e70","Type":"ContainerDied","Data":"fd788b5ea4d049faa5f6d184dcf520d83ac942368c726ef7d32f516a3fc146c0"}
Jul 29 18:30:03.115231 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:03.115203    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj"
Jul 29 18:30:04.000275 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:04.000245    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj"
Jul 29 18:30:04.000442 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:04.000256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-9hbwj" event={"ID":"94a9b25f-8529-46b8-8cef-d0a0644e3e70","Type":"ContainerDied","Data":"4d8353f3fd232c5c409a40c7ea93cd0efa6c2d7f518145c854b592f1e16b08ed"}
Jul 29 18:30:04.000442 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:04.000394    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d8353f3fd232c5c409a40c7ea93cd0efa6c2d7f518145c854b592f1e16b08ed"
Jul 29 18:30:23.307662 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.307630    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"]
Jul 29 18:30:23.308044 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.308001    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy"
Jul 29 18:30:23.308044 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.308012    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="227f8d5e-e4b2-4022-8e88-2f122805f0f6" containerName="istio-proxy"
Jul 29 18:30:23.308044 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.308022    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94a9b25f-8529-46b8-8cef-d0a0644e3e70" containerName="cleanup"
Jul 29 18:30:23.308044 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.308027    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="94a9b25f-8529-46b8-8cef-d0a0644e3e70" containerName="cleanup"
Jul 29 18:30:23.308194 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.308107    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="94a9b25f-8529-46b8-8cef-d0a0644e3e70" containerName="cleanup"
Jul 29 18:30:23.310287 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.310257    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.312627 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.312607    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:30:23.312739 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.312608    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8cd341-openshift-default-dockercfg-vhwwv\""
Jul 29 18:30:23.328630 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.328599    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"]
Jul 29 18:30:23.394028 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.393983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-certs\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262689f4-2f8b-437a-9861-39edcfbd2754-istio-podinfo\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-socket\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394139    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-data\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262689f4-2f8b-437a-9861-39edcfbd2754-istiod-ca-cert\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394187    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmcj4\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-kube-api-access-jmcj4\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394232 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394217    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-istio-token\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394541 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394304    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-credential-socket\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.394541 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.394332    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-envoy\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495355 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495314    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262689f4-2f8b-437a-9861-39edcfbd2754-istio-podinfo\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-socket\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-data\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262689f4-2f8b-437a-9861-39edcfbd2754-istiod-ca-cert\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495437    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmcj4\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-kube-api-access-jmcj4\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495482    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-istio-token\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495559 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-credential-socket\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495887 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495572    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-envoy\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495887 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-certs\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495998 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495883    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-socket\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495998 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-credential-socket\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.495998 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.495979    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-data\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.496161 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.496068    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-certs\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.496452 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.496431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262689f4-2f8b-437a-9861-39edcfbd2754-istiod-ca-cert\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.498019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.498000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-envoy\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.498128 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.498109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262689f4-2f8b-437a-9861-39edcfbd2754-istio-podinfo\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.503468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.503443    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmcj4\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-kube-api-access-jmcj4\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.503859 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.503841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-istio-token\") pod \"e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") " pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.621145 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.621059    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:23.759280 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.759235    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"]
Jul 29 18:30:23.761347 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:30:23.761323    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262689f4_2f8b_437a_9861_39edcfbd2754.slice/crio-9f1a1adb2c14de5fe6471effe52a9243e82753b9e6106d8f3c8408bf8f3c86ea WatchSource:0}: Error finding container 9f1a1adb2c14de5fe6471effe52a9243e82753b9e6106d8f3c8408bf8f3c86ea: Status 404 returned error can't find the container with id 9f1a1adb2c14de5fe6471effe52a9243e82753b9e6106d8f3c8408bf8f3c86ea
Jul 29 18:30:23.763405 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.763376    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:23.763484 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.763435    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:23.763484 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:23.763462    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:24.101366 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:24.101325    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" event={"ID":"262689f4-2f8b-437a-9861-39edcfbd2754","Type":"ContainerStarted","Data":"30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada"}
Jul 29 18:30:24.101581 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:24.101389    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" event={"ID":"262689f4-2f8b-437a-9861-39edcfbd2754","Type":"ContainerStarted","Data":"9f1a1adb2c14de5fe6471effe52a9243e82753b9e6106d8f3c8408bf8f3c86ea"}
Jul 29 18:30:24.123044 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:24.122990    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podStartSLOduration=1.122974082 podStartE2EDuration="1.122974082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:30:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:30:24.120937605 +0000 UTC m=+2214.362898666" watchObservedRunningTime="2026-07-29 18:30:24.122974082 +0000 UTC m=+2214.364935191"
Jul 29 18:30:24.621904 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:24.621866    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:24.626545 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:24.626519    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:25.106673 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:25.106639    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:25.107583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:25.107561    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:27.124650 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.124601    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"]
Jul 29 18:30:27.128046 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.128020    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.131021 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.130998    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-8cd341-serving-cert\""
Jul 29 18:30:27.144875 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.144836    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"]
Jul 29 18:30:27.230585 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.230544    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.230822 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.230745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.231073 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.231040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r845\" (UniqueName: \"kubernetes.io/projected/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-kube-api-access-7r845\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.331795 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.331749    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r845\" (UniqueName: \"kubernetes.io/projected/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-kube-api-access-7r845\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.332003 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.331880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.332003 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.331911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.332003 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:30:27.331983    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-serving-cert: secret "maas-api-e2e-inf-a-8cd341-serving-cert" not found
Jul 29 18:30:27.332183 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:30:27.332059    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls podName:b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2 nodeName:}" failed. No retries permitted until 2026-07-29 18:30:27.832038519 +0000 UTC m=+2218.073999537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls") pod "maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4" (UID: "b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2") : secret "maas-api-e2e-inf-a-8cd341-serving-cert" not found
Jul 29 18:30:27.332523 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.332504    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.343833 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.343808    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r845\" (UniqueName: \"kubernetes.io/projected/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-kube-api-access-7r845\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.838182 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.838147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:27.840921 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:27.840891    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls\") pod \"maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:28.039741 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:28.039700    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:28.214778 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:28.214743    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"]
Jul 29 18:30:28.219374 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:30:28.219339    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b85f02_83ce_4665_8f4b_84e5bd9fe3d2.slice/crio-e814c9eaa4d0ea66cfcfa555a7c13da93f7c8b5e1d2c7af06069691f6707ed0c WatchSource:0}: Error finding container e814c9eaa4d0ea66cfcfa555a7c13da93f7c8b5e1d2c7af06069691f6707ed0c: Status 404 returned error can't find the container with id e814c9eaa4d0ea66cfcfa555a7c13da93f7c8b5e1d2c7af06069691f6707ed0c
Jul 29 18:30:29.141798 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:29.141765    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4" event={"ID":"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2","Type":"ContainerStarted","Data":"34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387"}
Jul 29 18:30:29.141987 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:29.141815    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4" event={"ID":"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2","Type":"ContainerStarted","Data":"e814c9eaa4d0ea66cfcfa555a7c13da93f7c8b5e1d2c7af06069691f6707ed0c"}
Jul 29 18:30:29.141987 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:29.141843    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:29.161034 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:29.160967    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4" podStartSLOduration=1.7237430150000002 podStartE2EDuration="2.16094949s" totalImagesPullingTime="437.206475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:30:29.159445803 +0000 UTC m=+2219.401406836" watchObservedRunningTime="2026-07-29 18:30:29.16094949 +0000 UTC m=+2219.402910528"
Jul 29 18:30:30.464362 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:30.464322    2560 scope.go:117] "RemoveContainer" containerID="6f0bdb20fcaf34725fbc7a3405e593e605e5ab2538709831a95c52af79c1fa6b"
Jul 29 18:30:35.152941 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:35.152914    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:30:41.147462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:41.147400    2560 patch_prober.go:28] interesting pod/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 18:30:41.147972 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:41.147468    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 18:30:42.209216 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:42.209181    2560 generic.go:356] "Generic (PLEG): container finished" podID="262689f4-2f8b-437a-9861-39edcfbd2754" containerID="30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada" exitCode=0
Jul 29 18:30:42.209769 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:42.209288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" event={"ID":"262689f4-2f8b-437a-9861-39edcfbd2754","Type":"ContainerDied","Data":"30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada"}
Jul 29 18:30:42.209858 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:42.209840    2560 scope.go:117] "RemoveContainer" containerID="30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada"
Jul 29 18:30:42.210854 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:42.210829    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:42.210931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:42.210899    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:42.210931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:42.210927    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:43.219008 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:43.218974    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" event={"ID":"262689f4-2f8b-437a-9861-39edcfbd2754","Type":"ContainerStarted","Data":"b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055"}
Jul 29 18:30:43.622131 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:43.622032    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:43.622385 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:43.622338    2560 patch_prober.go:28] interesting pod/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 18:30:43.622466 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:43.622388    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 18:30:44.622726 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:44.621669    2560 patch_prober.go:28] interesting pod/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 18:30:44.622726 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:44.621736    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 18:30:45.622492 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:45.622445    2560 patch_prober.go:28] interesting pod/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 18:30:45.622717 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:45.622529    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 18:30:46.621945 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:46.621905    2560 patch_prober.go:28] interesting pod/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 18:30:46.622573 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:46.622535    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 18:30:47.624980 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:47.624946    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:47.625387 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:47.625222    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:47.625948 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:47.625930    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:30:51.602733 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.602650    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"]
Jul 29 18:30:51.605852 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.605829    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.608156 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.608137    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e6623d-openshift-default-dockercfg-c45wg\""
Jul 29 18:30:51.628154 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.628124    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"]
Jul 29 18:30:51.692763 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.692717    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-token\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.692956 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.692780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49fbf\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-kube-api-access-49fbf\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.692956 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.692876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-podinfo\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.692956 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.692916    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-certs\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.693123 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.692983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-socket\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.693123 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.693024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9673965-716e-45aa-ae3c-469ea40ead7a-istiod-ca-cert\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.693123 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.693065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-envoy\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.693221 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.693136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-data\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.693221 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.693174    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-credential-socket\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.793705 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.793667    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-podinfo\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.793705 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.793709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-certs\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.793954 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.793835    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-socket\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.793954 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.793881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9673965-716e-45aa-ae3c-469ea40ead7a-istiod-ca-cert\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.793954 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.793916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-envoy\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.793954 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.793938    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-data\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794166 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.793970    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-credential-socket\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794166 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.794018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-token\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794166 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.794075    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49fbf\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-kube-api-access-49fbf\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794166 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.794144    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-certs\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794417 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.794379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-socket\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794474 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.794423    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-data\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794539 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.794482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-credential-socket\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.794810 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.794789    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9673965-716e-45aa-ae3c-469ea40ead7a-istiod-ca-cert\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.796423 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.796399    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-envoy\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.796524 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.796474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-podinfo\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.802108 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.802084    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49fbf\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-kube-api-access-49fbf\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.802197 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.802089    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-token\") pod \"e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") " pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:51.916490 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:51.916405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:52.054758 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.054735    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"]
Jul 29 18:30:52.056865 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:30:52.056837    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9673965_716e_45aa_ae3c_469ea40ead7a.slice/crio-7818ae9d99184de68c93a0e1ce84aca86f785cd6d09827fa15439de648620fa8 WatchSource:0}: Error finding container 7818ae9d99184de68c93a0e1ce84aca86f785cd6d09827fa15439de648620fa8: Status 404 returned error can't find the container with id 7818ae9d99184de68c93a0e1ce84aca86f785cd6d09827fa15439de648620fa8
Jul 29 18:30:52.059657 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.059623    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:52.059747 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.059696    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:52.059747 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.059724    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:30:52.286514 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.286476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" event={"ID":"f9673965-716e-45aa-ae3c-469ea40ead7a","Type":"ContainerStarted","Data":"3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19"}
Jul 29 18:30:52.286729 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.286543    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" event={"ID":"f9673965-716e-45aa-ae3c-469ea40ead7a","Type":"ContainerStarted","Data":"7818ae9d99184de68c93a0e1ce84aca86f785cd6d09827fa15439de648620fa8"}
Jul 29 18:30:52.317284 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.317214    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podStartSLOduration=1.317195076 podStartE2EDuration="1.317195076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:30:52.314137162 +0000 UTC m=+2242.556098205" watchObservedRunningTime="2026-07-29 18:30:52.317195076 +0000 UTC m=+2242.559156113"
Jul 29 18:30:52.917093 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:52.917049    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:53.921754 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:53.921722    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:54.292152 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:54.292120    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:54.293240 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:54.293217    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:30:55.399522 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.399490    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"]
Jul 29 18:30:55.402227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.402207    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.405241 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.405218    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e6623d-serving-cert\""
Jul 29 18:30:55.414750 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.414729    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"]
Jul 29 18:30:55.534451 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.534411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/683b1884-5282-43c2-80e6-e13a947402aa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.534646 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.534492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qtcr\" (UniqueName: \"kubernetes.io/projected/683b1884-5282-43c2-80e6-e13a947402aa-kube-api-access-5qtcr\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.534646 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.534550    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.635909 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.635877    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qtcr\" (UniqueName: \"kubernetes.io/projected/683b1884-5282-43c2-80e6-e13a947402aa-kube-api-access-5qtcr\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.636084 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.635936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.636084 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.636010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/683b1884-5282-43c2-80e6-e13a947402aa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.636158 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:30:55.636106    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-serving-cert: secret "maas-api-e2e-inf-b-e6623d-serving-cert" not found
Jul 29 18:30:55.636198 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:30:55.636187    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls podName:683b1884-5282-43c2-80e6-e13a947402aa nodeName:}" failed. No retries permitted until 2026-07-29 18:30:56.136169178 +0000 UTC m=+2246.378130200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls") pod "maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn" (UID: "683b1884-5282-43c2-80e6-e13a947402aa") : secret "maas-api-e2e-inf-b-e6623d-serving-cert" not found
Jul 29 18:30:55.636661 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.636634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/683b1884-5282-43c2-80e6-e13a947402aa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:55.644089 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:55.644065    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qtcr\" (UniqueName: \"kubernetes.io/projected/683b1884-5282-43c2-80e6-e13a947402aa-kube-api-access-5qtcr\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:56.141659 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:56.141622    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:56.144147 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:56.144125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:56.314505 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:56.314476    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:56.451250 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:56.451222    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"]
Jul 29 18:30:56.452818 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:30:56.452792    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683b1884_5282_43c2_80e6_e13a947402aa.slice/crio-21905be7825ef57a24f78c2965e4a2d559b8e952963999bb7d697a79fdb62288 WatchSource:0}: Error finding container 21905be7825ef57a24f78c2965e4a2d559b8e952963999bb7d697a79fdb62288: Status 404 returned error can't find the container with id 21905be7825ef57a24f78c2965e4a2d559b8e952963999bb7d697a79fdb62288
Jul 29 18:30:57.307288 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:57.307228    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn" event={"ID":"683b1884-5282-43c2-80e6-e13a947402aa","Type":"ContainerStarted","Data":"3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b"}
Jul 29 18:30:57.307459 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:57.307318    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:30:57.307459 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:57.307349    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn" event={"ID":"683b1884-5282-43c2-80e6-e13a947402aa","Type":"ContainerStarted","Data":"21905be7825ef57a24f78c2965e4a2d559b8e952963999bb7d697a79fdb62288"}
Jul 29 18:30:57.325101 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:30:57.325043    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn" podStartSLOduration=1.9416318559999999 podStartE2EDuration="2.325026661s" totalImagesPullingTime="383.394805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:30:57.322875623 +0000 UTC m=+2247.564836651" watchObservedRunningTime="2026-07-29 18:30:57.325026661 +0000 UTC m=+2247.566987698"
Jul 29 18:31:03.325670 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:03.325625    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:31:10.226746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.226547    2560 patch_prober.go:28] interesting pod/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:41808->10.134.0.56:15021: read: connection reset by peer" start-of-body=
Jul 29 18:31:10.226746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.226609    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:41808->10.134.0.56:15021: read: connection reset by peer"
Jul 29 18:31:10.379772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.379739    2560 generic.go:356] "Generic (PLEG): container finished" podID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerID="3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19" exitCode=0
Jul 29 18:31:10.379928 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.379786    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" event={"ID":"f9673965-716e-45aa-ae3c-469ea40ead7a","Type":"ContainerDied","Data":"3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19"}
Jul 29 18:31:10.380200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.380187    2560 scope.go:117] "RemoveContainer" containerID="3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19"
Jul 29 18:31:10.381353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.381324    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:31:10.381419 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.381399    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:31:10.381460 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:10.381433    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:31:11.386455 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:11.386418    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" event={"ID":"f9673965-716e-45aa-ae3c-469ea40ead7a","Type":"ContainerStarted","Data":"c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a"}
Jul 29 18:31:11.928293 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:11.923756    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:31:11.928293 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:11.926378    2560 patch_prober.go:28] interesting pod/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 29 18:31:11.928293 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:11.926433    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 29 18:31:12.916893 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:12.916859    2560 patch_prober.go:28] interesting pod/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 29 18:31:12.917500 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:12.917470    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 29 18:31:13.917579 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:13.917532    2560 patch_prober.go:28] interesting pod/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 29 18:31:13.918042 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:13.917606    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 29 18:31:14.917318 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:14.917252    2560 patch_prober.go:28] interesting pod/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 29 18:31:14.917532 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:14.917347    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 29 18:31:15.920332 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:15.920302    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:31:15.920718 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:15.920613    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:31:15.921217 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:31:15.921202    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:33:05.812297 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:05.812245    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"]
Jul 29 18:33:05.973316 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:05.970901    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4" podUID="b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" containerName="maas-api" containerID="cri-o://34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387" gracePeriod=30
Jul 29 18:33:06.227142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.227115    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:33:06.347493 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.347400    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r845\" (UniqueName: \"kubernetes.io/projected/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-kube-api-access-7r845\") pod \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") "
Jul 29 18:33:06.347493 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.347461    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls\") pod \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") "
Jul 29 18:33:06.347725 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.347511    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-openshift-service-ca\") pod \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\" (UID: \"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2\") "
Jul 29 18:33:06.347883 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.347858    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" (UID: "b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:33:06.349771 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.349744    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" (UID: "b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:33:06.349869 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.349802    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-kube-api-access-7r845" (OuterVolumeSpecName: "kube-api-access-7r845") pod "b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" (UID: "b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2"). InnerVolumeSpecName "kube-api-access-7r845". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:33:06.448774 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.448738    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r845\" (UniqueName: \"kubernetes.io/projected/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-kube-api-access-7r845\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:06.448774 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.448769    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:06.448774 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.448780    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:06.977644 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.977611    2560 generic.go:356] "Generic (PLEG): container finished" podID="b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" containerID="34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387" exitCode=0
Jul 29 18:33:06.978103 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.977658    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"
Jul 29 18:33:06.978103 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.977663    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4" event={"ID":"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2","Type":"ContainerDied","Data":"34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387"}
Jul 29 18:33:06.978103 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.977770    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4" event={"ID":"b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2","Type":"ContainerDied","Data":"e814c9eaa4d0ea66cfcfa555a7c13da93f7c8b5e1d2c7af06069691f6707ed0c"}
Jul 29 18:33:06.978103 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.977794    2560 scope.go:117] "RemoveContainer" containerID="34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387"
Jul 29 18:33:06.987843 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.987824    2560 scope.go:117] "RemoveContainer" containerID="34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387"
Jul 29 18:33:06.988134 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:06.988112    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387\": container with ID starting with 34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387 not found: ID does not exist" containerID="34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387"
Jul 29 18:33:06.988198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.988147    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387"} err="failed to get container status \"34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387\": rpc error: code = NotFound desc = could not find container \"34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387\": container with ID starting with 34f71c9fb22ea1eefa94a7e94e456b9b4f9ef156b3920656c19b38c63035d387 not found: ID does not exist"
Jul 29 18:33:06.998684 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:06.998651    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"]
Jul 29 18:33:07.002765 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:07.002743    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8cd341-84c887c689-n6dq4"]
Jul 29 18:33:08.281679 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:08.281646    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" path="/var/lib/kubelet/pods/b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2/volumes"
Jul 29 18:33:12.633496 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:12.633460    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"]
Jul 29 18:33:12.634013 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:12.633646    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy" containerID="cri-o://b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055" gracePeriod=30
Jul 29 18:33:17.627391 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.627295    2560 patch_prober.go:28] interesting pod/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:33:17.627391 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.627383    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:33:17.885585 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.885511    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:33:17.961820 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.961783    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-istio-token\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.961835    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-credential-socket\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.961857    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-certs\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.961883    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262689f4-2f8b-437a-9861-39edcfbd2754-istio-podinfo\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962007    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-socket\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962251 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962095    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:17.962251 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962139    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262689f4-2f8b-437a-9861-39edcfbd2754-istiod-ca-cert\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962251 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962229    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmcj4\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-kube-api-access-jmcj4\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962459 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962287    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:17.962459 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962295    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:17.962459 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962307    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-data\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962459 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962330    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-envoy\") pod \"262689f4-2f8b-437a-9861-39edcfbd2754\" (UID: \"262689f4-2f8b-437a-9861-39edcfbd2754\") "
Jul 29 18:33:17.962667 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962559    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/262689f4-2f8b-437a-9861-39edcfbd2754-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:33:17.962896 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962877    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:17.962943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962903    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:17.962943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962920    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:17.962943 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.962934    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/262689f4-2f8b-437a-9861-39edcfbd2754-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:17.963461 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.963437    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-data" (OuterVolumeSpecName: "istio-data") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:17.964369 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.964341    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/262689f4-2f8b-437a-9861-39edcfbd2754-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:33:17.964477 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.964440    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-kube-api-access-jmcj4" (OuterVolumeSpecName: "kube-api-access-jmcj4") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "kube-api-access-jmcj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:33:17.964806 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.964783    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-istio-token" (OuterVolumeSpecName: "istio-token") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:33:17.964806 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:17.964795    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "262689f4-2f8b-437a-9861-39edcfbd2754" (UID: "262689f4-2f8b-437a-9861-39edcfbd2754"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:18.039820 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.039787    2560 generic.go:356] "Generic (PLEG): container finished" podID="262689f4-2f8b-437a-9861-39edcfbd2754" containerID="b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055" exitCode=0
Jul 29 18:33:18.040017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.039835    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"
Jul 29 18:33:18.040017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.039869    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" event={"ID":"262689f4-2f8b-437a-9861-39edcfbd2754","Type":"ContainerDied","Data":"b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055"}
Jul 29 18:33:18.040017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.039957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm" event={"ID":"262689f4-2f8b-437a-9861-39edcfbd2754","Type":"ContainerDied","Data":"9f1a1adb2c14de5fe6471effe52a9243e82753b9e6106d8f3c8408bf8f3c86ea"}
Jul 29 18:33:18.040017 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.039989    2560 scope.go:117] "RemoveContainer" containerID="b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055"
Jul 29 18:33:18.049871 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.049854    2560 scope.go:117] "RemoveContainer" containerID="30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada"
Jul 29 18:33:18.058133 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.058115    2560 scope.go:117] "RemoveContainer" containerID="b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055"
Jul 29 18:33:18.058578 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:18.058555    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055\": container with ID starting with b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055 not found: ID does not exist" containerID="b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055"
Jul 29 18:33:18.058645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.058587    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055"} err="failed to get container status \"b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055\": rpc error: code = NotFound desc = could not find container \"b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055\": container with ID starting with b1d230dbf0e9af0dbfb64d46cffe373555ff7991d4c671d30aed9ca1346ee055 not found: ID does not exist"
Jul 29 18:33:18.058645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.058605    2560 scope.go:117] "RemoveContainer" containerID="30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada"
Jul 29 18:33:18.058868 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:18.058850    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada\": container with ID starting with 30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada not found: ID does not exist" containerID="30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada"
Jul 29 18:33:18.058918 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.058873    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada"} err="failed to get container status \"30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada\": rpc error: code = NotFound desc = could not find container \"30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada\": container with ID starting with 30b4451be71f85c57af7d409982f0f25f3e2d1352dc9e267b2c570af623a1ada not found: ID does not exist"
Jul 29 18:33:18.062591 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.062567    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"]
Jul 29 18:33:18.063701 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.063685    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:18.063746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.063704    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/262689f4-2f8b-437a-9861-39edcfbd2754-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:18.063746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.063712    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmcj4\" (UniqueName: \"kubernetes.io/projected/262689f4-2f8b-437a-9861-39edcfbd2754-kube-api-access-jmcj4\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:18.063746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.063723    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:18.063746 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.063731    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/262689f4-2f8b-437a-9861-39edcfbd2754-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:18.067042 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.067021    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8cd341-openshift-default-675dc59fd4-p46qm"]
Jul 29 18:33:18.282611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:18.282582    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" path="/var/lib/kubelet/pods/262689f4-2f8b-437a-9861-39edcfbd2754/volumes"
Jul 29 18:33:23.507763 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:23.507727    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"]
Jul 29 18:33:23.508434 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:23.507966    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn" podUID="683b1884-5282-43c2-80e6-e13a947402aa" containerName="maas-api" containerID="cri-o://3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b" gracePeriod=30
Jul 29 18:33:23.965944 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:23.965919    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:33:24.071229 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.071140    2560 generic.go:356] "Generic (PLEG): container finished" podID="683b1884-5282-43c2-80e6-e13a947402aa" containerID="3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b" exitCode=0
Jul 29 18:33:24.071229 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.071185    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn" event={"ID":"683b1884-5282-43c2-80e6-e13a947402aa","Type":"ContainerDied","Data":"3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b"}
Jul 29 18:33:24.071229 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.071228    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn" event={"ID":"683b1884-5282-43c2-80e6-e13a947402aa","Type":"ContainerDied","Data":"21905be7825ef57a24f78c2965e4a2d559b8e952963999bb7d697a79fdb62288"}
Jul 29 18:33:24.071458 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.071252    2560 scope.go:117] "RemoveContainer" containerID="3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b"
Jul 29 18:33:24.071458 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.071186    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"
Jul 29 18:33:24.080718 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.080699    2560 scope.go:117] "RemoveContainer" containerID="3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b"
Jul 29 18:33:24.081068 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:24.081046    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b\": container with ID starting with 3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b not found: ID does not exist" containerID="3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b"
Jul 29 18:33:24.081139 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.081077    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b"} err="failed to get container status \"3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b\": rpc error: code = NotFound desc = could not find container \"3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b\": container with ID starting with 3d9cf703306a2226458e64d20b111745a522b3b93db069af120c0b34e054a08b not found: ID does not exist"
Jul 29 18:33:24.124313 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.124257    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qtcr\" (UniqueName: \"kubernetes.io/projected/683b1884-5282-43c2-80e6-e13a947402aa-kube-api-access-5qtcr\") pod \"683b1884-5282-43c2-80e6-e13a947402aa\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") "
Jul 29 18:33:24.124473 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.124356    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls\") pod \"683b1884-5282-43c2-80e6-e13a947402aa\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") "
Jul 29 18:33:24.124473 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.124433    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/683b1884-5282-43c2-80e6-e13a947402aa-openshift-service-ca\") pod \"683b1884-5282-43c2-80e6-e13a947402aa\" (UID: \"683b1884-5282-43c2-80e6-e13a947402aa\") "
Jul 29 18:33:24.124772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.124743    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/683b1884-5282-43c2-80e6-e13a947402aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "683b1884-5282-43c2-80e6-e13a947402aa" (UID: "683b1884-5282-43c2-80e6-e13a947402aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:33:24.126604 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.126577    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683b1884-5282-43c2-80e6-e13a947402aa-kube-api-access-5qtcr" (OuterVolumeSpecName: "kube-api-access-5qtcr") pod "683b1884-5282-43c2-80e6-e13a947402aa" (UID: "683b1884-5282-43c2-80e6-e13a947402aa"). InnerVolumeSpecName "kube-api-access-5qtcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:33:24.126721 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.126701    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "683b1884-5282-43c2-80e6-e13a947402aa" (UID: "683b1884-5282-43c2-80e6-e13a947402aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:33:24.226253 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.226219    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/683b1884-5282-43c2-80e6-e13a947402aa-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:24.226253 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.226251    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/683b1884-5282-43c2-80e6-e13a947402aa-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:24.226253 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.226261    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qtcr\" (UniqueName: \"kubernetes.io/projected/683b1884-5282-43c2-80e6-e13a947402aa-kube-api-access-5qtcr\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:24.386965 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.386886    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"]
Jul 29 18:33:24.390218 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:24.390189    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e6623d-66c5d95889-dcldn"]
Jul 29 18:33:26.281595 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:26.281553    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="683b1884-5282-43c2-80e6-e13a947402aa" path="/var/lib/kubelet/pods/683b1884-5282-43c2-80e6-e13a947402aa/volumes"
Jul 29 18:33:30.356615 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:30.356589    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:33:30.361132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:30.361109    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:33:30.467482 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:30.467441    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"]
Jul 29 18:33:30.467716 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:30.467675    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy" containerID="cri-o://c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a" gracePeriod=30
Jul 29 18:33:30.923367 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:30.923329    2560 patch_prober.go:28] interesting pod/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:33:30.923544 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:30.923386    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:33:35.304742 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.304710    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"]
Jul 29 18:33:35.305149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305137    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" containerName="maas-api"
Jul 29 18:33:35.305198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305151    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" containerName="maas-api"
Jul 29 18:33:35.305198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305164    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy"
Jul 29 18:33:35.305198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305171    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy"
Jul 29 18:33:35.305198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305177    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy"
Jul 29 18:33:35.305198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305183    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy"
Jul 29 18:33:35.305198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305189    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683b1884-5282-43c2-80e6-e13a947402aa" containerName="maas-api"
Jul 29 18:33:35.305198 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305194    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="683b1884-5282-43c2-80e6-e13a947402aa" containerName="maas-api"
Jul 29 18:33:35.305433 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305258    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy"
Jul 29 18:33:35.305433 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305302    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="262689f4-2f8b-437a-9861-39edcfbd2754" containerName="istio-proxy"
Jul 29 18:33:35.305433 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305311    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9b85f02-83ce-4665-8f4b-84e5bd9fe3d2" containerName="maas-api"
Jul 29 18:33:35.305433 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.305320    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="683b1884-5282-43c2-80e6-e13a947402aa" containerName="maas-api"
Jul 29 18:33:35.308350 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.308327    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.310449 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.310428    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-45bfe9-serving-cert\""
Jul 29 18:33:35.321496 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.321474    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"]
Jul 29 18:33:35.326474 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.326448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln7wc\" (UniqueName: \"kubernetes.io/projected/805463f3-8daf-46aa-88a2-12da5ec8f9d7-kube-api-access-ln7wc\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.326610 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.326559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805463f3-8daf-46aa-88a2-12da5ec8f9d7-openshift-service-ca\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.326610 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.326583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.427586 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.427549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805463f3-8daf-46aa-88a2-12da5ec8f9d7-openshift-service-ca\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.427586 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.427591    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.427801 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.427649    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln7wc\" (UniqueName: \"kubernetes.io/projected/805463f3-8daf-46aa-88a2-12da5ec8f9d7-kube-api-access-ln7wc\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.427801 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:35.427770    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-serving-cert: secret "maas-api-e2e-api-a-45bfe9-serving-cert" not found
Jul 29 18:33:35.427919 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:35.427896    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls podName:805463f3-8daf-46aa-88a2-12da5ec8f9d7 nodeName:}" failed. No retries permitted until 2026-07-29 18:33:35.92787254 +0000 UTC m=+2406.169833578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls") pod "maas-api-e2e-api-a-45bfe9-78d97fd6-djz75" (UID: "805463f3-8daf-46aa-88a2-12da5ec8f9d7") : secret "maas-api-e2e-api-a-45bfe9-serving-cert" not found
Jul 29 18:33:35.428311 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.428295    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805463f3-8daf-46aa-88a2-12da5ec8f9d7-openshift-service-ca\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.437966 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.437936    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln7wc\" (UniqueName: \"kubernetes.io/projected/805463f3-8daf-46aa-88a2-12da5ec8f9d7-kube-api-access-ln7wc\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.717536 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.717508    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:33:35.731503 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731478    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-envoy\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731523    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-token\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731549    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-data\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731578    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-socket\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731609    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-podinfo\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731646    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-certs\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731913 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731670    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-credential-socket\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731913 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731734    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49fbf\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-kube-api-access-49fbf\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.731913 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731820    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9673965-716e-45aa-ae3c-469ea40ead7a-istiod-ca-cert\") pod \"f9673965-716e-45aa-ae3c-469ea40ead7a\" (UID: \"f9673965-716e-45aa-ae3c-469ea40ead7a\") "
Jul 29 18:33:35.732468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.731814    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:35.732468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.732065    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:35.732468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.732082    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:35.732468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.732367    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9673965-716e-45aa-ae3c-469ea40ead7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:33:35.732751 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.732583    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:35.734078 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.734054    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:33:35.734305 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.734277    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-kube-api-access-49fbf" (OuterVolumeSpecName: "kube-api-access-49fbf") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "kube-api-access-49fbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:33:35.734403 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.734335    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:33:35.734456 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.734410    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9673965-716e-45aa-ae3c-469ea40ead7a" (UID: "f9673965-716e-45aa-ae3c-469ea40ead7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:33:35.833019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.832985    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49fbf\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-kube-api-access-49fbf\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833011    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9673965-716e-45aa-ae3c-469ea40ead7a-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833022    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833030    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833037    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833046    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833055    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9673965-716e-45aa-ae3c-469ea40ead7a-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833063    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.833338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.833071    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9673965-716e-45aa-ae3c-469ea40ead7a-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:33:35.933693 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.933605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:35.936360 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:35.936334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls\") pod \"maas-api-e2e-api-a-45bfe9-78d97fd6-djz75\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:36.137335 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.137298    2560 generic.go:356] "Generic (PLEG): container finished" podID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerID="c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a" exitCode=0
Jul 29 18:33:36.137528 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.137348    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"
Jul 29 18:33:36.137528 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.137429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" event={"ID":"f9673965-716e-45aa-ae3c-469ea40ead7a","Type":"ContainerDied","Data":"c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a"}
Jul 29 18:33:36.137528 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.137496    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r" event={"ID":"f9673965-716e-45aa-ae3c-469ea40ead7a","Type":"ContainerDied","Data":"7818ae9d99184de68c93a0e1ce84aca86f785cd6d09827fa15439de648620fa8"}
Jul 29 18:33:36.137684 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.137530    2560 scope.go:117] "RemoveContainer" containerID="c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a"
Jul 29 18:33:36.147104 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.147085    2560 scope.go:117] "RemoveContainer" containerID="3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19"
Jul 29 18:33:36.155910 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.155883    2560 scope.go:117] "RemoveContainer" containerID="c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a"
Jul 29 18:33:36.156374 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:36.156342    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a\": container with ID starting with c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a not found: ID does not exist" containerID="c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a"
Jul 29 18:33:36.157107 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.156386    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a"} err="failed to get container status \"c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a\": rpc error: code = NotFound desc = could not find container \"c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a\": container with ID starting with c15f5d313c6228f0aee7ab1f60980e590c378b73185d20cd0056f03e6727312a not found: ID does not exist"
Jul 29 18:33:36.157107 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.156411    2560 scope.go:117] "RemoveContainer" containerID="3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19"
Jul 29 18:33:36.157107 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:33:36.156807    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19\": container with ID starting with 3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19 not found: ID does not exist" containerID="3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19"
Jul 29 18:33:36.157107 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.156836    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19"} err="failed to get container status \"3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19\": rpc error: code = NotFound desc = could not find container \"3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19\": container with ID starting with 3474142df20317d741e266aeceeb191ea0e73639e15448f965a5a16a93169e19 not found: ID does not exist"
Jul 29 18:33:36.159054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.159006    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"]
Jul 29 18:33:36.165003 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.164981    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e6623d-openshift-default-7d76496dd8-slb9r"]
Jul 29 18:33:36.218468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.218378    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:36.286181 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.286145    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" path="/var/lib/kubelet/pods/f9673965-716e-45aa-ae3c-469ea40ead7a/volumes"
Jul 29 18:33:36.358633 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.358603    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"]
Jul 29 18:33:36.361640 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:33:36.361610    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805463f3_8daf_46aa_88a2_12da5ec8f9d7.slice/crio-35b8ea914c035112dc0452cb2c7cea0ee771ba86c00d003f5cef5c0b7000ba5b WatchSource:0}: Error finding container 35b8ea914c035112dc0452cb2c7cea0ee771ba86c00d003f5cef5c0b7000ba5b: Status 404 returned error can't find the container with id 35b8ea914c035112dc0452cb2c7cea0ee771ba86c00d003f5cef5c0b7000ba5b
Jul 29 18:33:36.362995 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:36.362978    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:33:37.143974 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:37.143876    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75" event={"ID":"805463f3-8daf-46aa-88a2-12da5ec8f9d7","Type":"ContainerStarted","Data":"98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627"}
Jul 29 18:33:37.143974 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:37.143948    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:37.144196 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:37.143981    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75" event={"ID":"805463f3-8daf-46aa-88a2-12da5ec8f9d7","Type":"ContainerStarted","Data":"35b8ea914c035112dc0452cb2c7cea0ee771ba86c00d003f5cef5c0b7000ba5b"}
Jul 29 18:33:37.160546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:37.160486    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75" podStartSLOduration=1.735437681 podStartE2EDuration="2.160467205s" totalImagesPullingTime="425.029524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:33:37.159194883 +0000 UTC m=+2407.401155919" watchObservedRunningTime="2026-07-29 18:33:37.160467205 +0000 UTC m=+2407.402428244"
Jul 29 18:33:43.155922 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:43.155889    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:33:59.811617 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.811539    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"]
Jul 29 18:33:59.812069 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.812052    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy"
Jul 29 18:33:59.812116 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.812073    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy"
Jul 29 18:33:59.812116 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.812086    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy"
Jul 29 18:33:59.812116 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.812091    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy"
Jul 29 18:33:59.812210 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.812165    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy"
Jul 29 18:33:59.815824 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.815808    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.819178 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.819155    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:33:59.819671 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.819654    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a19316-openshift-default-dockercfg-j26bm\""
Jul 29 18:33:59.839212 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.839187    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"]
Jul 29 18:33:59.972575 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972531    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d75df4-c307-4c4a-a928-874f685bf314-istiod-ca-cert\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972749 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972617    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-socket\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972749 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d75df4-c307-4c4a-a928-874f685bf314-istio-podinfo\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972749 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972679    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-istio-token\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972749 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972710    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-credential-socket\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972888 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-data\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972888 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66rnt\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-kube-api-access-66rnt\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-certs\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:33:59.972959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:33:59.972929    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-envoy\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074187    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d75df4-c307-4c4a-a928-874f685bf314-istiod-ca-cert\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074226    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-socket\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d75df4-c307-4c4a-a928-874f685bf314-istio-podinfo\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074308 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074260    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-istio-token\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-credential-socket\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-data\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66rnt\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-kube-api-access-66rnt\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-certs\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074645 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074613    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-envoy\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074691    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-credential-socket\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.074931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074877    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-socket\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.075036 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074925    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-data\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.075036 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.074999    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-certs\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.075224 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.075206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d75df4-c307-4c4a-a928-874f685bf314-istiod-ca-cert\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.076877 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.076850    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-envoy\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.077051 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.077029    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d75df4-c307-4c4a-a928-874f685bf314-istio-podinfo\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.083274 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.083244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-istio-token\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.083461 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.083442    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66rnt\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-kube-api-access-66rnt\") pod \"e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") " pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.127385 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.127346    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:00.275582 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.275556    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"]
Jul 29 18:34:00.278138 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:34:00.278105    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d75df4_c307_4c4a_a928_874f685bf314.slice/crio-ca7d7ce47be4d717b610952f3a06314ea2984a1c6f73174ef42e93d4e440dd8f WatchSource:0}: Error finding container ca7d7ce47be4d717b610952f3a06314ea2984a1c6f73174ef42e93d4e440dd8f: Status 404 returned error can't find the container with id ca7d7ce47be4d717b610952f3a06314ea2984a1c6f73174ef42e93d4e440dd8f
Jul 29 18:34:00.280159 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.280124    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:34:00.280248 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.280203    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:34:00.280248 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:00.280232    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:34:01.264444 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:01.264411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" event={"ID":"e4d75df4-c307-4c4a-a928-874f685bf314","Type":"ContainerStarted","Data":"022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f"}
Jul 29 18:34:01.264829 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:01.264498    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" event={"ID":"e4d75df4-c307-4c4a-a928-874f685bf314","Type":"ContainerStarted","Data":"ca7d7ce47be4d717b610952f3a06314ea2984a1c6f73174ef42e93d4e440dd8f"}
Jul 29 18:34:01.286582 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:01.286522    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" podStartSLOduration=2.286506692 podStartE2EDuration="2.286506692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:01.284414716 +0000 UTC m=+2431.526375767" watchObservedRunningTime="2026-07-29 18:34:01.286506692 +0000 UTC m=+2431.528467804"
Jul 29 18:34:02.128373 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:02.128332    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:02.133302 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:02.133251    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:02.270597 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:02.270562    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:02.271513 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:02.271496    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:04.383987 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.383951    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"]
Jul 29 18:34:04.384821 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.384799    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9673965-716e-45aa-ae3c-469ea40ead7a" containerName="istio-proxy"
Jul 29 18:34:04.388315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.388292    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.390673 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.390655    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a19316-serving-cert\""
Jul 29 18:34:04.397512 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.397484    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"]
Jul 29 18:34:04.415245 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.415206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-openshift-service-ca\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.415452 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.415252    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb9zs\" (UniqueName: \"kubernetes.io/projected/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-kube-api-access-wb9zs\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.415452 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.415303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-maas-api-tls\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.516450 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.516418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-openshift-service-ca\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.516655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.516470    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb9zs\" (UniqueName: \"kubernetes.io/projected/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-kube-api-access-wb9zs\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.516655 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.516500    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-maas-api-tls\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.517315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.517287    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-openshift-service-ca\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.519967 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.519940    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-maas-api-tls\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.525253 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.525221    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb9zs\" (UniqueName: \"kubernetes.io/projected/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-kube-api-access-wb9zs\") pod \"maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.701593 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.701499    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:04.842416 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:04.842393    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"]
Jul 29 18:34:04.845103 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:34:04.845074    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35bbdc2_3629_44f1_9eb7_cb2b17d53b01.slice/crio-83b5e045d3d7ed30aa375d87bdb8e07e7d10f957345729cb7b5e2170255d7d3e WatchSource:0}: Error finding container 83b5e045d3d7ed30aa375d87bdb8e07e7d10f957345729cb7b5e2170255d7d3e: Status 404 returned error can't find the container with id 83b5e045d3d7ed30aa375d87bdb8e07e7d10f957345729cb7b5e2170255d7d3e
Jul 29 18:34:05.286623 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:05.286587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt" event={"ID":"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01","Type":"ContainerStarted","Data":"87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008"}
Jul 29 18:34:05.286850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:05.286637    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt" event={"ID":"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01","Type":"ContainerStarted","Data":"83b5e045d3d7ed30aa375d87bdb8e07e7d10f957345729cb7b5e2170255d7d3e"}
Jul 29 18:34:05.286850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:05.286713    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:05.303790 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:05.303727    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt" podStartSLOduration=0.96387369 podStartE2EDuration="1.303711681s" totalImagesPullingTime="339.837991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:05.302643335 +0000 UTC m=+2435.544604374" watchObservedRunningTime="2026-07-29 18:34:05.303711681 +0000 UTC m=+2435.545672719"
Jul 29 18:34:11.297839 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:11.297811    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:34:20.369945 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:20.369860    2560 generic.go:356] "Generic (PLEG): container finished" podID="e4d75df4-c307-4c4a-a928-874f685bf314" containerID="022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f" exitCode=0
Jul 29 18:34:20.370349 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:20.369935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" event={"ID":"e4d75df4-c307-4c4a-a928-874f685bf314","Type":"ContainerDied","Data":"022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f"}
Jul 29 18:34:20.370398 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:20.370354    2560 scope.go:117] "RemoveContainer" containerID="022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f"
Jul 29 18:34:20.371478 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:20.371454    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:34:20.371545 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:20.371535    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:34:20.371582 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:20.371575    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:34:21.377867 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:21.377832    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" event={"ID":"e4d75df4-c307-4c4a-a928-874f685bf314","Type":"ContainerStarted","Data":"9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab"}
Jul 29 18:34:22.128467 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:22.128422    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:22.128754 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:22.128724    2560 patch_prober.go:28] interesting pod/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 29 18:34:22.128846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:22.128808    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 29 18:34:23.128608 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:23.128567    2560 patch_prober.go:28] interesting pod/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 29 18:34:23.129224 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:23.129191    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 29 18:34:24.128169 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:24.128121    2560 patch_prober.go:28] interesting pod/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 29 18:34:24.128411 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:24.128204    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 29 18:34:25.128436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:25.128388    2560 patch_prober.go:28] interesting pod/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 29 18:34:25.128959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:25.128482    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 29 18:34:26.131085 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:26.131048    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:26.131550 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:26.131336    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:26.131986 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:26.131968    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:34:36.098966 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.098929    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"]
Jul 29 18:34:36.103459 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.103424    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"
Jul 29 18:34:36.105568 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.105539    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:34:36.110589 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.110564    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"]
Jul 29 18:34:36.247461 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.247425    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34d9adcb-7b74-48f1-b28d-2f22fa789da1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66\" (UID: \"34d9adcb-7b74-48f1-b28d-2f22fa789da1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"
Jul 29 18:34:36.348925 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.348887    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34d9adcb-7b74-48f1-b28d-2f22fa789da1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66\" (UID: \"34d9adcb-7b74-48f1-b28d-2f22fa789da1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"
Jul 29 18:34:36.349431 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.349383    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34d9adcb-7b74-48f1-b28d-2f22fa789da1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66\" (UID: \"34d9adcb-7b74-48f1-b28d-2f22fa789da1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"
Jul 29 18:34:36.415685 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.415652    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"
Jul 29 18:34:36.553795 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:36.553771    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"]
Jul 29 18:34:36.555923 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:34:36.555886    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d9adcb_7b74_48f1_b28d_2f22fa789da1.slice/crio-0d4f2ef75a0cb799b541358a48db1e429e2853a46599a212f332b1d394d982e0 WatchSource:0}: Error finding container 0d4f2ef75a0cb799b541358a48db1e429e2853a46599a212f332b1d394d982e0: Status 404 returned error can't find the container with id 0d4f2ef75a0cb799b541358a48db1e429e2853a46599a212f332b1d394d982e0
Jul 29 18:34:37.461117 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:37.461074    2560 generic.go:356] "Generic (PLEG): container finished" podID="34d9adcb-7b74-48f1-b28d-2f22fa789da1" containerID="f53c6694252e5ccbbd302d3f09ab9421e330c0b58ce39f0fab7c40ba4621f12b" exitCode=0
Jul 29 18:34:37.461555 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:37.461158    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66" event={"ID":"34d9adcb-7b74-48f1-b28d-2f22fa789da1","Type":"ContainerDied","Data":"f53c6694252e5ccbbd302d3f09ab9421e330c0b58ce39f0fab7c40ba4621f12b"}
Jul 29 18:34:37.461555 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:37.461208    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66" event={"ID":"34d9adcb-7b74-48f1-b28d-2f22fa789da1","Type":"ContainerStarted","Data":"0d4f2ef75a0cb799b541358a48db1e429e2853a46599a212f332b1d394d982e0"}
Jul 29 18:34:38.601287 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:38.601239    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"
Jul 29 18:34:38.672384 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:38.672353    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34d9adcb-7b74-48f1-b28d-2f22fa789da1-maas-api-service-ca\") pod \"34d9adcb-7b74-48f1-b28d-2f22fa789da1\" (UID: \"34d9adcb-7b74-48f1-b28d-2f22fa789da1\") "
Jul 29 18:34:38.672724 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:38.672703    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34d9adcb-7b74-48f1-b28d-2f22fa789da1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "34d9adcb-7b74-48f1-b28d-2f22fa789da1" (UID: "34d9adcb-7b74-48f1-b28d-2f22fa789da1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:34:38.773853 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:38.773816    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34d9adcb-7b74-48f1-b28d-2f22fa789da1-maas-api-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:34:39.473546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:39.473510    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66" event={"ID":"34d9adcb-7b74-48f1-b28d-2f22fa789da1","Type":"ContainerDied","Data":"0d4f2ef75a0cb799b541358a48db1e429e2853a46599a212f332b1d394d982e0"}
Jul 29 18:34:39.473546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:39.473543    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"
Jul 29 18:34:39.473758 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:39.473565    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d4f2ef75a0cb799b541358a48db1e429e2853a46599a212f332b1d394d982e0"
Jul 29 18:34:51.646163 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:51.646125    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"]
Jul 29 18:34:51.646703 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:51.646360    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75" podUID="805463f3-8daf-46aa-88a2-12da5ec8f9d7" containerName="maas-api" containerID="cri-o://98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627" gracePeriod=30
Jul 29 18:34:51.851779 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:51.851743    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"]
Jul 29 18:34:51.857284 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:51.857239    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-45bfe9-gjf66"]
Jul 29 18:34:51.906612 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:51.906545    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:34:52.004081 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.004042    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln7wc\" (UniqueName: \"kubernetes.io/projected/805463f3-8daf-46aa-88a2-12da5ec8f9d7-kube-api-access-ln7wc\") pod \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") "
Jul 29 18:34:52.004081 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.004082    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls\") pod \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") "
Jul 29 18:34:52.004361 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.004189    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805463f3-8daf-46aa-88a2-12da5ec8f9d7-openshift-service-ca\") pod \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\" (UID: \"805463f3-8daf-46aa-88a2-12da5ec8f9d7\") "
Jul 29 18:34:52.004580 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.004556    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805463f3-8daf-46aa-88a2-12da5ec8f9d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "805463f3-8daf-46aa-88a2-12da5ec8f9d7" (UID: "805463f3-8daf-46aa-88a2-12da5ec8f9d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:34:52.006334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.006309    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "805463f3-8daf-46aa-88a2-12da5ec8f9d7" (UID: "805463f3-8daf-46aa-88a2-12da5ec8f9d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:34:52.006436 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.006396    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805463f3-8daf-46aa-88a2-12da5ec8f9d7-kube-api-access-ln7wc" (OuterVolumeSpecName: "kube-api-access-ln7wc") pod "805463f3-8daf-46aa-88a2-12da5ec8f9d7" (UID: "805463f3-8daf-46aa-88a2-12da5ec8f9d7"). InnerVolumeSpecName "kube-api-access-ln7wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:34:52.105309 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.105252    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805463f3-8daf-46aa-88a2-12da5ec8f9d7-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:34:52.105309 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.105305    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln7wc\" (UniqueName: \"kubernetes.io/projected/805463f3-8daf-46aa-88a2-12da5ec8f9d7-kube-api-access-ln7wc\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:34:52.105309 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.105317    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805463f3-8daf-46aa-88a2-12da5ec8f9d7-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:34:52.286414 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.286378    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34d9adcb-7b74-48f1-b28d-2f22fa789da1" path="/var/lib/kubelet/pods/34d9adcb-7b74-48f1-b28d-2f22fa789da1/volumes"
Jul 29 18:34:52.541496 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.541402    2560 generic.go:356] "Generic (PLEG): container finished" podID="805463f3-8daf-46aa-88a2-12da5ec8f9d7" containerID="98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627" exitCode=0
Jul 29 18:34:52.541496 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.541471    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"
Jul 29 18:34:52.541693 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.541488    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75" event={"ID":"805463f3-8daf-46aa-88a2-12da5ec8f9d7","Type":"ContainerDied","Data":"98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627"}
Jul 29 18:34:52.541693 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.541576    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75" event={"ID":"805463f3-8daf-46aa-88a2-12da5ec8f9d7","Type":"ContainerDied","Data":"35b8ea914c035112dc0452cb2c7cea0ee771ba86c00d003f5cef5c0b7000ba5b"}
Jul 29 18:34:52.541693 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.541605    2560 scope.go:117] "RemoveContainer" containerID="98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627"
Jul 29 18:34:52.550687 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.550662    2560 scope.go:117] "RemoveContainer" containerID="98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627"
Jul 29 18:34:52.550937 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:34:52.550914    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627\": container with ID starting with 98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627 not found: ID does not exist" containerID="98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627"
Jul 29 18:34:52.551001 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.550945    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627"} err="failed to get container status \"98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627\": rpc error: code = NotFound desc = could not find container \"98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627\": container with ID starting with 98fe21090086c8387875bb8e0fc66bb4538cc8a525833b07d36598a80a6b8627 not found: ID does not exist"
Jul 29 18:34:52.558212 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.558186    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"]
Jul 29 18:34:52.563902 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:52.563881    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45bfe9-78d97fd6-djz75"]
Jul 29 18:34:54.282546 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:34:54.282517    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805463f3-8daf-46aa-88a2-12da5ec8f9d7" path="/var/lib/kubelet/pods/805463f3-8daf-46aa-88a2-12da5ec8f9d7/volumes"
Jul 29 18:35:07.916864 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:07.916823    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"]
Jul 29 18:35:07.917677 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:07.917641    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt" podUID="f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" containerName="maas-api" containerID="cri-o://87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008" gracePeriod=30
Jul 29 18:35:08.184353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.184291    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:35:08.363509 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.363480    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb9zs\" (UniqueName: \"kubernetes.io/projected/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-kube-api-access-wb9zs\") pod \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") "
Jul 29 18:35:08.363690 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.363536    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-openshift-service-ca\") pod \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") "
Jul 29 18:35:08.363690 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.363641    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-maas-api-tls\") pod \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\" (UID: \"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01\") "
Jul 29 18:35:08.364019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.363966    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" (UID: "f35bbdc2-3629-44f1-9eb7-cb2b17d53b01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:35:08.365783 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.365755    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-kube-api-access-wb9zs" (OuterVolumeSpecName: "kube-api-access-wb9zs") pod "f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" (UID: "f35bbdc2-3629-44f1-9eb7-cb2b17d53b01"). InnerVolumeSpecName "kube-api-access-wb9zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:35:08.365783 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.365769    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" (UID: "f35bbdc2-3629-44f1-9eb7-cb2b17d53b01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:35:08.464621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.464533    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:08.464621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.464563    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:08.464621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.464575    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb9zs\" (UniqueName: \"kubernetes.io/projected/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01-kube-api-access-wb9zs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:08.624508 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.624471    2560 generic.go:356] "Generic (PLEG): container finished" podID="f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" containerID="87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008" exitCode=0
Jul 29 18:35:08.624690 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.624517    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"
Jul 29 18:35:08.624690 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.624515    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt" event={"ID":"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01","Type":"ContainerDied","Data":"87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008"}
Jul 29 18:35:08.624690 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.624657    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt" event={"ID":"f35bbdc2-3629-44f1-9eb7-cb2b17d53b01","Type":"ContainerDied","Data":"83b5e045d3d7ed30aa375d87bdb8e07e7d10f957345729cb7b5e2170255d7d3e"}
Jul 29 18:35:08.624817 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.624690    2560 scope.go:117] "RemoveContainer" containerID="87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008"
Jul 29 18:35:08.634221 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.634203    2560 scope.go:117] "RemoveContainer" containerID="87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008"
Jul 29 18:35:08.634541 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:35:08.634522    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008\": container with ID starting with 87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008 not found: ID does not exist" containerID="87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008"
Jul 29 18:35:08.634614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.634550    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008"} err="failed to get container status \"87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008\": rpc error: code = NotFound desc = could not find container \"87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008\": container with ID starting with 87d1e0df97187b9b8a689ce1d9fa7ea63b0ef4bf5ef90354609f182209a87008 not found: ID does not exist"
Jul 29 18:35:08.657058 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.656970    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"]
Jul 29 18:35:08.659224 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:08.659202    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a19316-6c8d67cb4f-52hxt"]
Jul 29 18:35:10.286457 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:10.284642    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" path="/var/lib/kubelet/pods/f35bbdc2-3629-44f1-9eb7-cb2b17d53b01/volumes"
Jul 29 18:35:14.763939 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:14.763907    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"]
Jul 29 18:35:14.764338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:14.764118    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy" containerID="cri-o://9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab" gracePeriod=30
Jul 29 18:35:15.598872 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.598834    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"]
Jul 29 18:35:15.599248 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599236    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805463f3-8daf-46aa-88a2-12da5ec8f9d7" containerName="maas-api"
Jul 29 18:35:15.599304 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599250    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="805463f3-8daf-46aa-88a2-12da5ec8f9d7" containerName="maas-api"
Jul 29 18:35:15.599304 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599283    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34d9adcb-7b74-48f1-b28d-2f22fa789da1" containerName="revoke-keys"
Jul 29 18:35:15.599304 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599289    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="34d9adcb-7b74-48f1-b28d-2f22fa789da1" containerName="revoke-keys"
Jul 29 18:35:15.599304 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599300    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" containerName="maas-api"
Jul 29 18:35:15.599438 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599307    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" containerName="maas-api"
Jul 29 18:35:15.599438 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599368    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="34d9adcb-7b74-48f1-b28d-2f22fa789da1" containerName="revoke-keys"
Jul 29 18:35:15.599438 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599380    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="805463f3-8daf-46aa-88a2-12da5ec8f9d7" containerName="maas-api"
Jul 29 18:35:15.599438 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.599386    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35bbdc2-3629-44f1-9eb7-cb2b17d53b01" containerName="maas-api"
Jul 29 18:35:15.603718 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.603701    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.605857 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.605835    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-297c4d-openshift-default-dockercfg-4qnpj\""
Jul 29 18:35:15.614753 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.614729    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"]
Jul 29 18:35:15.634428 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1806f65e-f061-4f46-9e45-071c6fd0d942-istiod-ca-cert\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634577 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634454    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-certs\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634577 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-envoy\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634577 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634515    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6gb6\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-kube-api-access-f6gb6\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634577 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634541    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-data\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634716 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-socket\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634716 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-credential-socket\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634716 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634650    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-podinfo\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.634716 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.634703    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-token\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735143 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735109    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-token\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735152    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1806f65e-f061-4f46-9e45-071c6fd0d942-istiod-ca-cert\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-certs\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-envoy\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735239    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6gb6\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-kube-api-access-f6gb6\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-data\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735309    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-socket\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735350    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-credential-socket\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735382 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735384    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-podinfo\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735808 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735725    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-certs\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735869 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-data\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735869 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735855    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-credential-socket\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735973 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735921    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-socket\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.735973 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.735960    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1806f65e-f061-4f46-9e45-071c6fd0d942-istiod-ca-cert\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.737883 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.737866    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-envoy\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.737960 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.737944    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-podinfo\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.743649 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.743625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-token\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.743817 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.743799    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6gb6\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-kube-api-access-f6gb6\") pod \"e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") " pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:15.914477 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:15.914380    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:16.054100 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.054076    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"]
Jul 29 18:35:16.056033 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:35:16.055996    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1806f65e_f061_4f46_9e45_071c6fd0d942.slice/crio-bef6d4fe63f33e05a21a3bc419f2b00f830b3764cc3845c905631cb4caed70ff WatchSource:0}: Error finding container bef6d4fe63f33e05a21a3bc419f2b00f830b3764cc3845c905631cb4caed70ff: Status 404 returned error can't find the container with id bef6d4fe63f33e05a21a3bc419f2b00f830b3764cc3845c905631cb4caed70ff
Jul 29 18:35:16.058087 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.058049    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:35:16.058173 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.058142    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:35:16.058215 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.058184    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:35:16.669557 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.669517    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" event={"ID":"1806f65e-f061-4f46-9e45-071c6fd0d942","Type":"ContainerStarted","Data":"20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219"}
Jul 29 18:35:16.669745 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.669584    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" event={"ID":"1806f65e-f061-4f46-9e45-071c6fd0d942","Type":"ContainerStarted","Data":"bef6d4fe63f33e05a21a3bc419f2b00f830b3764cc3845c905631cb4caed70ff"}
Jul 29 18:35:16.696666 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.696609    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" podStartSLOduration=1.696592919 podStartE2EDuration="1.696592919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:16.694558683 +0000 UTC m=+2506.936519722" watchObservedRunningTime="2026-07-29 18:35:16.696592919 +0000 UTC m=+2506.938553956"
Jul 29 18:35:16.914514 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:16.914478    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:17.919498 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:17.919467    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:18.679152 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:18.679116    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:18.680100 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:18.680079    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:19.873368 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:19.873335    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"]
Jul 29 18:35:19.877492 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:19.877467    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:19.879710 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:19.879688    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-297c4d-serving-cert\""
Jul 29 18:35:19.889078 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:19.889013    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"]
Jul 29 18:35:19.984080 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:19.984044    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w6gw\" (UniqueName: \"kubernetes.io/projected/c061dbd6-6581-4e28-b9f7-abf336927ff6-kube-api-access-6w6gw\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:19.984244 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:19.984093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061dbd6-6581-4e28-b9f7-abf336927ff6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:19.984324 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:19.984303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061dbd6-6581-4e28-b9f7-abf336927ff6-maas-api-tls\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.035709 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.035649    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:35:20.084760 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.084730    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-data\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.084949 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.084771    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d75df4-c307-4c4a-a928-874f685bf314-istiod-ca-cert\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.084949 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.084809    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-envoy\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.084949 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.084834    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-socket\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.084949 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.084870    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66rnt\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-kube-api-access-66rnt\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.084949 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.084927    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-credential-socket\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.085223 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.084956    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d75df4-c307-4c4a-a928-874f685bf314-istio-podinfo\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.085223 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085008    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-certs\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.085223 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085036    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-istio-token\") pod \"e4d75df4-c307-4c4a-a928-874f685bf314\" (UID: \"e4d75df4-c307-4c4a-a928-874f685bf314\") "
Jul 29 18:35:20.085223 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085163    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:35:20.085223 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085191    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061dbd6-6581-4e28-b9f7-abf336927ff6-maas-api-tls\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.085506 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085245    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w6gw\" (UniqueName: \"kubernetes.io/projected/c061dbd6-6581-4e28-b9f7-abf336927ff6-kube-api-access-6w6gw\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.085506 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061dbd6-6581-4e28-b9f7-abf336927ff6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.085609 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085584    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.085821 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085238    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:35:20.085821 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085463    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:35:20.085821 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085524    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4d75df4-c307-4c4a-a928-874f685bf314-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:35:20.085821 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.085761    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:35:20.086482 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.086456    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061dbd6-6581-4e28-b9f7-abf336927ff6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.087371 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.087340    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:35:20.087784 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.087756    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-kube-api-access-66rnt" (OuterVolumeSpecName: "kube-api-access-66rnt") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "kube-api-access-66rnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:35:20.087890 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.087873    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:35:20.087931 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.087919    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061dbd6-6581-4e28-b9f7-abf336927ff6-maas-api-tls\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.088114 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.088087    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4d75df4-c307-4c4a-a928-874f685bf314-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4d75df4-c307-4c4a-a928-874f685bf314" (UID: "e4d75df4-c307-4c4a-a928-874f685bf314"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:35:20.093560 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.093535    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w6gw\" (UniqueName: \"kubernetes.io/projected/c061dbd6-6581-4e28-b9f7-abf336927ff6-kube-api-access-6w6gw\") pod \"maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187061    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187092    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187101    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187108    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4d75df4-c307-4c4a-a928-874f685bf314-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187116    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187124    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66rnt\" (UniqueName: \"kubernetes.io/projected/e4d75df4-c307-4c4a-a928-874f685bf314-kube-api-access-66rnt\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187134    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4d75df4-c307-4c4a-a928-874f685bf314-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.187142 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.187143    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4d75df4-c307-4c4a-a928-874f685bf314-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:35:20.190917 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.190887    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:20.342639 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.342609    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"]
Jul 29 18:35:20.344873 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:35:20.344844    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc061dbd6_6581_4e28_b9f7_abf336927ff6.slice/crio-c421ab22bee73424682c64c08c1de0c101af83c85c2b6f1fb68707398bffbe7c WatchSource:0}: Error finding container c421ab22bee73424682c64c08c1de0c101af83c85c2b6f1fb68707398bffbe7c: Status 404 returned error can't find the container with id c421ab22bee73424682c64c08c1de0c101af83c85c2b6f1fb68707398bffbe7c
Jul 29 18:35:20.691167 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.691070    2560 generic.go:356] "Generic (PLEG): container finished" podID="e4d75df4-c307-4c4a-a928-874f685bf314" containerID="9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab" exitCode=0
Jul 29 18:35:20.691167 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.691112    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"
Jul 29 18:35:20.691167 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.691109    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" event={"ID":"e4d75df4-c307-4c4a-a928-874f685bf314","Type":"ContainerDied","Data":"9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab"}
Jul 29 18:35:20.691167 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.691166    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c" event={"ID":"e4d75df4-c307-4c4a-a928-874f685bf314","Type":"ContainerDied","Data":"ca7d7ce47be4d717b610952f3a06314ea2984a1c6f73174ef42e93d4e440dd8f"}
Jul 29 18:35:20.691500 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.691188    2560 scope.go:117] "RemoveContainer" containerID="9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab"
Jul 29 18:35:20.692708 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.692685    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2" event={"ID":"c061dbd6-6581-4e28-b9f7-abf336927ff6","Type":"ContainerStarted","Data":"c421ab22bee73424682c64c08c1de0c101af83c85c2b6f1fb68707398bffbe7c"}
Jul 29 18:35:20.701393 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.701380    2560 scope.go:117] "RemoveContainer" containerID="022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f"
Jul 29 18:35:20.706358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.706333    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"]
Jul 29 18:35:20.710239 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.710220    2560 scope.go:117] "RemoveContainer" containerID="9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab"
Jul 29 18:35:20.710601 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:35:20.710566    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab\": container with ID starting with 9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab not found: ID does not exist" containerID="9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab"
Jul 29 18:35:20.710700 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.710606    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab"} err="failed to get container status \"9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab\": rpc error: code = NotFound desc = could not find container \"9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab\": container with ID starting with 9e1c740707d16e60dfab22f9158195f2c2dac3091aed214635dfaac8cc9b0fab not found: ID does not exist"
Jul 29 18:35:20.710700 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.710632    2560 scope.go:117] "RemoveContainer" containerID="022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f"
Jul 29 18:35:20.710988 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:35:20.710962    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f\": container with ID starting with 022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f not found: ID does not exist" containerID="022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f"
Jul 29 18:35:20.711130 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.711106    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f"} err="failed to get container status \"022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f\": rpc error: code = NotFound desc = could not find container \"022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f\": container with ID starting with 022a906bcf0481becd56696a37dac7b7099e1edf0af7724ae691d977cc5a007f not found: ID does not exist"
Jul 29 18:35:20.713837 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:20.713818    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a19316-openshift-default-7fc764d6d8-4gt7c"]
Jul 29 18:35:22.282181 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:22.282146    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" path="/var/lib/kubelet/pods/e4d75df4-c307-4c4a-a928-874f685bf314/volumes"
Jul 29 18:35:22.705893 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:22.705797    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2" event={"ID":"c061dbd6-6581-4e28-b9f7-abf336927ff6","Type":"ContainerStarted","Data":"33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c"}
Jul 29 18:35:22.705893 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:22.705866    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:22.723211 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:22.723159    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2" podStartSLOduration=2.325590882 podStartE2EDuration="3.723140882s" totalImagesPullingTime="1.39755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:22.721916 +0000 UTC m=+2512.963877039" watchObservedRunningTime="2026-07-29 18:35:22.723140882 +0000 UTC m=+2512.965101920"
Jul 29 18:35:28.726679 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:28.726586    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:35:34.026176 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.026143    2560 patch_prober.go:28] interesting pod/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": read tcp 10.134.0.2:56434->10.134.0.62:15021: read: connection reset by peer" start-of-body=
Jul 29 18:35:34.026606 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.026214    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": read tcp 10.134.0.2:56434->10.134.0.62:15021: read: connection reset by peer"
Jul 29 18:35:34.772485 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.772449    2560 generic.go:356] "Generic (PLEG): container finished" podID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerID="20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219" exitCode=0
Jul 29 18:35:34.772670 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.772519    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" event={"ID":"1806f65e-f061-4f46-9e45-071c6fd0d942","Type":"ContainerDied","Data":"20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219"}
Jul 29 18:35:34.772935 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.772922    2560 scope.go:117] "RemoveContainer" containerID="20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219"
Jul 29 18:35:34.774000 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.773971    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:35:34.774099 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.774052    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:35:34.774170 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:34.774099    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:35:35.780514 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:35.780482    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" event={"ID":"1806f65e-f061-4f46-9e45-071c6fd0d942","Type":"ContainerStarted","Data":"cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d"}
Jul 29 18:35:35.915227 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:35.915190    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:35.915491 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:35.915302    2560 patch_prober.go:28] interesting pod/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:35:35.915698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:35.915666    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 29 18:35:36.915730 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:36.915254    2560 patch_prober.go:28] interesting pod/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:35:36.915730 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:36.915339    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 29 18:35:37.915663 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:37.915617    2560 patch_prober.go:28] interesting pod/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:35:37.915869 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:37.915687    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 29 18:35:38.915414 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:38.915375    2560 patch_prober.go:28] interesting pod/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:35:38.915721 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:38.915688    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 29 18:35:39.918018 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:39.917982    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:39.918428 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:39.918378    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:39.919087 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:39.919070    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:35:47.882429 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.882395    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"]
Jul 29 18:35:47.882798 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.882784    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy"
Jul 29 18:35:47.882847 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.882799    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy"
Jul 29 18:35:47.882847 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.882817    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy"
Jul 29 18:35:47.882847 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.882822    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy"
Jul 29 18:35:47.882937 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.882887    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy"
Jul 29 18:35:47.882937 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.882897    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d75df4-c307-4c4a-a928-874f685bf314" containerName="istio-proxy"
Jul 29 18:35:47.886451 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.886429    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:47.888771 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.888751    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f9520e-serving-cert\""
Jul 29 18:35:47.902386 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:47.902358    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"]
Jul 29 18:35:48.059160 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.059127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkhwz\" (UniqueName: \"kubernetes.io/projected/f99c7104-600e-417a-a08d-2b282aed35f3-kube-api-access-kkhwz\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.059353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.059173    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.059353 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.059240    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f99c7104-600e-417a-a08d-2b282aed35f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.160065 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.159973    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.160065 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.160036    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f99c7104-600e-417a-a08d-2b282aed35f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.160348 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.160110    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkhwz\" (UniqueName: \"kubernetes.io/projected/f99c7104-600e-417a-a08d-2b282aed35f3-kube-api-access-kkhwz\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.160348 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:35:48.160158    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-serving-cert: secret "maas-api-e2e-shared-b-f9520e-serving-cert" not found
Jul 29 18:35:48.160348 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:35:48.160249    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls podName:f99c7104-600e-417a-a08d-2b282aed35f3 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:48.660227966 +0000 UTC m=+2538.902189002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls") pod "maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7" (UID: "f99c7104-600e-417a-a08d-2b282aed35f3") : secret "maas-api-e2e-shared-b-f9520e-serving-cert" not found
Jul 29 18:35:48.160784 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.160764    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f99c7104-600e-417a-a08d-2b282aed35f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.170289 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.170252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkhwz\" (UniqueName: \"kubernetes.io/projected/f99c7104-600e-417a-a08d-2b282aed35f3-kube-api-access-kkhwz\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.664611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.664568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.667132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.667102    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.797567 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.797533    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:48.948965 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:48.948934    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"]
Jul 29 18:35:48.950551 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:35:48.950527    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99c7104_600e_417a_a08d_2b282aed35f3.slice/crio-6d3804aea697b4463b4760eeb5efbb524029781f0ec3ffe6bc016875800c403b WatchSource:0}: Error finding container 6d3804aea697b4463b4760eeb5efbb524029781f0ec3ffe6bc016875800c403b: Status 404 returned error can't find the container with id 6d3804aea697b4463b4760eeb5efbb524029781f0ec3ffe6bc016875800c403b
Jul 29 18:35:49.852864 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:49.852831    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7" event={"ID":"f99c7104-600e-417a-a08d-2b282aed35f3","Type":"ContainerStarted","Data":"67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2"}
Jul 29 18:35:49.853063 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:49.852881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7" event={"ID":"f99c7104-600e-417a-a08d-2b282aed35f3","Type":"ContainerStarted","Data":"6d3804aea697b4463b4760eeb5efbb524029781f0ec3ffe6bc016875800c403b"}
Jul 29 18:35:49.853063 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:49.852917    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:35:49.869778 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:49.869729    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7" podStartSLOduration=2.463720215 podStartE2EDuration="2.869714442s" totalImagesPullingTime="405.994227ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:49.868395352 +0000 UTC m=+2540.110356391" watchObservedRunningTime="2026-07-29 18:35:49.869714442 +0000 UTC m=+2540.111675481"
Jul 29 18:35:55.870831 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:35:55.870802    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:38:30.394067 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:38:30.393988    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:38:30.399937 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:38:30.399918    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:41:30.598986 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:41:30.598907    2560 scope.go:117] "RemoveContainer" containerID="f53c6694252e5ccbbd302d3f09ab9421e330c0b58ce39f0fab7c40ba4621f12b"
Jul 29 18:43:30.429913 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:43:30.429883    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:43:30.436149 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:43:30.436129    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:45:00.136311 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.136259    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8"]
Jul 29 18:45:00.139681 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.139661    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8"
Jul 29 18:45:00.141669 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.141649    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:45:00.149016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.148989    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8"]
Jul 29 18:45:00.152295 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.152256    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8"
Jul 29 18:45:00.288609 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.288586    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8"]
Jul 29 18:45:00.290964 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:45:00.290932    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98110a33_9998_4622_8be8_6079012239c5.slice/crio-5ddd78275599523475eda9abab3be7d222d12f564853b5396a067092d443fbbc WatchSource:0}: Error finding container 5ddd78275599523475eda9abab3be7d222d12f564853b5396a067092d443fbbc: Status 404 returned error can't find the container with id 5ddd78275599523475eda9abab3be7d222d12f564853b5396a067092d443fbbc
Jul 29 18:45:00.659313 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.659246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8" event={"ID":"98110a33-9998-4622-8be8-6079012239c5","Type":"ContainerStarted","Data":"5fc9b90365f0cb422c5fee931ffd133fe1d75048b1e90b22935291ec5d38266b"}
Jul 29 18:45:00.659486 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.659333    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8" event={"ID":"98110a33-9998-4622-8be8-6079012239c5","Type":"ContainerStarted","Data":"5ddd78275599523475eda9abab3be7d222d12f564853b5396a067092d443fbbc"}
Jul 29 18:45:00.672169 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:00.672115    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8" podStartSLOduration=0.672098876 podStartE2EDuration="672.098876ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:00.671369042 +0000 UTC m=+3090.913330094" watchObservedRunningTime="2026-07-29 18:45:00.672098876 +0000 UTC m=+3090.914059913"
Jul 29 18:45:01.665885 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:01.665846    2560 generic.go:356] "Generic (PLEG): container finished" podID="98110a33-9998-4622-8be8-6079012239c5" containerID="5fc9b90365f0cb422c5fee931ffd133fe1d75048b1e90b22935291ec5d38266b" exitCode=0
Jul 29 18:45:01.666352 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:01.665929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8" event={"ID":"98110a33-9998-4622-8be8-6079012239c5","Type":"ContainerDied","Data":"5fc9b90365f0cb422c5fee931ffd133fe1d75048b1e90b22935291ec5d38266b"}
Jul 29 18:45:02.804940 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:02.804916    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8"
Jul 29 18:45:03.679647 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:03.679611    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8" event={"ID":"98110a33-9998-4622-8be8-6079012239c5","Type":"ContainerDied","Data":"5ddd78275599523475eda9abab3be7d222d12f564853b5396a067092d443fbbc"}
Jul 29 18:45:03.679647 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:03.679639    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-sdjb8"
Jul 29 18:45:03.679908 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:03.679658    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ddd78275599523475eda9abab3be7d222d12f564853b5396a067092d443fbbc"
Jul 29 18:45:24.148114 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.148076    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nd42p"]
Jul 29 18:45:24.148611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.148527    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98110a33-9998-4622-8be8-6079012239c5" containerName="cleanup"
Jul 29 18:45:24.148611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.148539    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="98110a33-9998-4622-8be8-6079012239c5" containerName="cleanup"
Jul 29 18:45:24.148611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.148609    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="98110a33-9998-4622-8be8-6079012239c5" containerName="cleanup"
Jul 29 18:45:24.151827 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.151806    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:24.153558 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.153537    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-tlq5f\""
Jul 29 18:45:24.157745 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.157721    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nd42p"]
Jul 29 18:45:24.192864 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.192831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmfm9\" (UniqueName: \"kubernetes.io/projected/6730cd31-4bb9-4dd9-9529-67cd0726fd96-kube-api-access-qmfm9\") pod \"kuadrant-operator-catalog-nd42p\" (UID: \"6730cd31-4bb9-4dd9-9529-67cd0726fd96\") " pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:24.294386 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.294349    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmfm9\" (UniqueName: \"kubernetes.io/projected/6730cd31-4bb9-4dd9-9529-67cd0726fd96-kube-api-access-qmfm9\") pod \"kuadrant-operator-catalog-nd42p\" (UID: \"6730cd31-4bb9-4dd9-9529-67cd0726fd96\") " pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:24.301754 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.301723    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmfm9\" (UniqueName: \"kubernetes.io/projected/6730cd31-4bb9-4dd9-9529-67cd0726fd96-kube-api-access-qmfm9\") pod \"kuadrant-operator-catalog-nd42p\" (UID: \"6730cd31-4bb9-4dd9-9529-67cd0726fd96\") " pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:24.463001 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.462896    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:24.608496 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.608472    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nd42p"]
Jul 29 18:45:24.610635 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:45:24.610607    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6730cd31_4bb9_4dd9_9529_67cd0726fd96.slice/crio-e77147f51a352efb94e3a73850aaac48ae1ad49093255de6b90502f1a9a5d103 WatchSource:0}: Error finding container e77147f51a352efb94e3a73850aaac48ae1ad49093255de6b90502f1a9a5d103: Status 404 returned error can't find the container with id e77147f51a352efb94e3a73850aaac48ae1ad49093255de6b90502f1a9a5d103
Jul 29 18:45:24.612006 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.611988    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:45:24.783047 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:24.783015    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nd42p" event={"ID":"6730cd31-4bb9-4dd9-9529-67cd0726fd96","Type":"ContainerStarted","Data":"e77147f51a352efb94e3a73850aaac48ae1ad49093255de6b90502f1a9a5d103"}
Jul 29 18:45:26.796104 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:26.796067    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nd42p" event={"ID":"6730cd31-4bb9-4dd9-9529-67cd0726fd96","Type":"ContainerStarted","Data":"8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d"}
Jul 29 18:45:26.810166 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:26.810104    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-nd42p" podStartSLOduration=1.271686692 podStartE2EDuration="2.810083705s" totalImagesPullingTime="1.538397013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:26.809957439 +0000 UTC m=+3117.051918477" watchObservedRunningTime="2026-07-29 18:45:26.810083705 +0000 UTC m=+3117.052044744"
Jul 29 18:45:34.463021 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:34.462981    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:34.463542 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:34.463042    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:34.485522 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:34.485487    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:34.858309 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:34.858278    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:34.891857 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:34.891823    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nd42p"]
Jul 29 18:45:36.844660 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:36.844607    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-nd42p" podUID="6730cd31-4bb9-4dd9-9529-67cd0726fd96" containerName="registry-server" containerID="cri-o://8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d" gracePeriod=2
Jul 29 18:45:37.094970 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.094912    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:37.222660 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.222619    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmfm9\" (UniqueName: \"kubernetes.io/projected/6730cd31-4bb9-4dd9-9529-67cd0726fd96-kube-api-access-qmfm9\") pod \"6730cd31-4bb9-4dd9-9529-67cd0726fd96\" (UID: \"6730cd31-4bb9-4dd9-9529-67cd0726fd96\") "
Jul 29 18:45:37.224866 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.224832    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6730cd31-4bb9-4dd9-9529-67cd0726fd96-kube-api-access-qmfm9" (OuterVolumeSpecName: "kube-api-access-qmfm9") pod "6730cd31-4bb9-4dd9-9529-67cd0726fd96" (UID: "6730cd31-4bb9-4dd9-9529-67cd0726fd96"). InnerVolumeSpecName "kube-api-access-qmfm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:45:37.324112 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.324072    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmfm9\" (UniqueName: \"kubernetes.io/projected/6730cd31-4bb9-4dd9-9529-67cd0726fd96-kube-api-access-qmfm9\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:45:37.850581 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.850550    2560 generic.go:356] "Generic (PLEG): container finished" podID="6730cd31-4bb9-4dd9-9529-67cd0726fd96" containerID="8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d" exitCode=0
Jul 29 18:45:37.851027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.850593    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nd42p"
Jul 29 18:45:37.851027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.850642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nd42p" event={"ID":"6730cd31-4bb9-4dd9-9529-67cd0726fd96","Type":"ContainerDied","Data":"8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d"}
Jul 29 18:45:37.851027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.850687    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nd42p" event={"ID":"6730cd31-4bb9-4dd9-9529-67cd0726fd96","Type":"ContainerDied","Data":"e77147f51a352efb94e3a73850aaac48ae1ad49093255de6b90502f1a9a5d103"}
Jul 29 18:45:37.851027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.850711    2560 scope.go:117] "RemoveContainer" containerID="8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d"
Jul 29 18:45:37.860641 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.860618    2560 scope.go:117] "RemoveContainer" containerID="8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d"
Jul 29 18:45:37.860879 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:45:37.860859    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d\": container with ID starting with 8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d not found: ID does not exist" containerID="8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d"
Jul 29 18:45:37.860941 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.860887    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d"} err="failed to get container status \"8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d\": rpc error: code = NotFound desc = could not find container \"8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d\": container with ID starting with 8ecdbf9179264e971db5ceae2d7d015ec0115278f4b53969ea8aaf8f41d30d0d not found: ID does not exist"
Jul 29 18:45:37.870339 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.870317    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nd42p"]
Jul 29 18:45:37.873457 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:37.873437    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nd42p"]
Jul 29 18:45:38.282194 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:38.282162    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6730cd31-4bb9-4dd9-9529-67cd0726fd96" path="/var/lib/kubelet/pods/6730cd31-4bb9-4dd9-9529-67cd0726fd96/volumes"
Jul 29 18:45:55.026466 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.026432    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7720-139837267975600"]
Jul 29 18:45:55.026878 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.026865    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6730cd31-4bb9-4dd9-9529-67cd0726fd96" containerName="registry-server"
Jul 29 18:45:55.026927 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.026879    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6730cd31-4bb9-4dd9-9529-67cd0726fd96" containerName="registry-server"
Jul 29 18:45:55.026963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.026953    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6730cd31-4bb9-4dd9-9529-67cd0726fd96" containerName="registry-server"
Jul 29 18:45:55.030128 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.030111    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267975600"
Jul 29 18:45:55.032121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.032091    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lxnjd\""
Jul 29 18:45:55.034490 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.034467    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837267975600"]
Jul 29 18:45:55.099007 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.098974    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78b8d\" (UniqueName: \"kubernetes.io/projected/2b18b672-8ade-4fe5-b013-3a1e2fcac83e-kube-api-access-78b8d\") pod \"test-curl-7720-139837267975600\" (UID: \"2b18b672-8ade-4fe5-b013-3a1e2fcac83e\") " pod="opendatahub/test-curl-7720-139837267975600"
Jul 29 18:45:55.200229 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.200196    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78b8d\" (UniqueName: \"kubernetes.io/projected/2b18b672-8ade-4fe5-b013-3a1e2fcac83e-kube-api-access-78b8d\") pod \"test-curl-7720-139837267975600\" (UID: \"2b18b672-8ade-4fe5-b013-3a1e2fcac83e\") " pod="opendatahub/test-curl-7720-139837267975600"
Jul 29 18:45:55.207893 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.207864    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78b8d\" (UniqueName: \"kubernetes.io/projected/2b18b672-8ade-4fe5-b013-3a1e2fcac83e-kube-api-access-78b8d\") pod \"test-curl-7720-139837267975600\" (UID: \"2b18b672-8ade-4fe5-b013-3a1e2fcac83e\") " pod="opendatahub/test-curl-7720-139837267975600"
Jul 29 18:45:55.340643 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.340553    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267975600"
Jul 29 18:45:55.474242 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.474201    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837267975600"]
Jul 29 18:45:55.477404 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:45:55.477373    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b18b672_8ade_4fe5_b013_3a1e2fcac83e.slice/crio-5a85c45db2a3060ce9780230b122135aee821fd16598115b7da5b2f531a4cab1 WatchSource:0}: Error finding container 5a85c45db2a3060ce9780230b122135aee821fd16598115b7da5b2f531a4cab1: Status 404 returned error can't find the container with id 5a85c45db2a3060ce9780230b122135aee821fd16598115b7da5b2f531a4cab1
Jul 29 18:45:55.942734 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:55.942689    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837267975600" event={"ID":"2b18b672-8ade-4fe5-b013-3a1e2fcac83e","Type":"ContainerStarted","Data":"5a85c45db2a3060ce9780230b122135aee821fd16598115b7da5b2f531a4cab1"}
Jul 29 18:45:56.949429 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:56.949399    2560 generic.go:356] "Generic (PLEG): container finished" podID="2b18b672-8ade-4fe5-b013-3a1e2fcac83e" containerID="0c5868b71d82c84ad7fefc8225cf06e627178b60315d84c86d8d92a48e5a62c0" exitCode=0
Jul 29 18:45:56.949773 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:56.949476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837267975600" event={"ID":"2b18b672-8ade-4fe5-b013-3a1e2fcac83e","Type":"ContainerDied","Data":"0c5868b71d82c84ad7fefc8225cf06e627178b60315d84c86d8d92a48e5a62c0"}
Jul 29 18:45:58.087755 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.087730    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267975600"
Jul 29 18:45:58.122211 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.122180    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7720-139837267975600_2b18b672-8ade-4fe5-b013-3a1e2fcac83e/test-curl-7720-139837267975600/0.log"
Jul 29 18:45:58.129472 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.129448    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78b8d\" (UniqueName: \"kubernetes.io/projected/2b18b672-8ade-4fe5-b013-3a1e2fcac83e-kube-api-access-78b8d\") pod \"2b18b672-8ade-4fe5-b013-3a1e2fcac83e\" (UID: \"2b18b672-8ade-4fe5-b013-3a1e2fcac83e\") "
Jul 29 18:45:58.131631 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.131605    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b18b672-8ade-4fe5-b013-3a1e2fcac83e-kube-api-access-78b8d" (OuterVolumeSpecName: "kube-api-access-78b8d") pod "2b18b672-8ade-4fe5-b013-3a1e2fcac83e" (UID: "2b18b672-8ade-4fe5-b013-3a1e2fcac83e"). InnerVolumeSpecName "kube-api-access-78b8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:45:58.143190 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.143167    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7720-139837267975600"]
Jul 29 18:45:58.145642 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.145620    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7720-139837267975600"]
Jul 29 18:45:58.230818 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.230735    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78b8d\" (UniqueName: \"kubernetes.io/projected/2b18b672-8ade-4fe5-b013-3a1e2fcac83e-kube-api-access-78b8d\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:45:58.260164 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.260133    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7720-139837267989296"]
Jul 29 18:45:58.260551 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.260537    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b18b672-8ade-4fe5-b013-3a1e2fcac83e" containerName="test-curl-7720-139837267975600"
Jul 29 18:45:58.260611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.260553    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b18b672-8ade-4fe5-b013-3a1e2fcac83e" containerName="test-curl-7720-139837267975600"
Jul 29 18:45:58.260659 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.260646    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b18b672-8ade-4fe5-b013-3a1e2fcac83e" containerName="test-curl-7720-139837267975600"
Jul 29 18:45:58.263846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.263831    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:45:58.267596 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.267569    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837267989296"]
Jul 29 18:45:58.282359 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.282334    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b18b672-8ade-4fe5-b013-3a1e2fcac83e" path="/var/lib/kubelet/pods/2b18b672-8ade-4fe5-b013-3a1e2fcac83e/volumes"
Jul 29 18:45:58.332318 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.332256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c95dn\" (UniqueName: \"kubernetes.io/projected/88446146-c743-4a14-94a1-6fb8d0de5d05-kube-api-access-c95dn\") pod \"test-curl-7720-139837267989296\" (UID: \"88446146-c743-4a14-94a1-6fb8d0de5d05\") " pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:45:58.433403 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.433369    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c95dn\" (UniqueName: \"kubernetes.io/projected/88446146-c743-4a14-94a1-6fb8d0de5d05-kube-api-access-c95dn\") pod \"test-curl-7720-139837267989296\" (UID: \"88446146-c743-4a14-94a1-6fb8d0de5d05\") " pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:45:58.440288 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.440247    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c95dn\" (UniqueName: \"kubernetes.io/projected/88446146-c743-4a14-94a1-6fb8d0de5d05-kube-api-access-c95dn\") pod \"test-curl-7720-139837267989296\" (UID: \"88446146-c743-4a14-94a1-6fb8d0de5d05\") " pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:45:58.574809 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.574768    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:45:58.712279 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.712244    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837267989296"]
Jul 29 18:45:58.715173 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:45:58.715145    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88446146_c743_4a14_94a1_6fb8d0de5d05.slice/crio-9c434125fa07a09ed06d3733b37e686ee7423407dbd227a897688b6f6ec273ef WatchSource:0}: Error finding container 9c434125fa07a09ed06d3733b37e686ee7423407dbd227a897688b6f6ec273ef: Status 404 returned error can't find the container with id 9c434125fa07a09ed06d3733b37e686ee7423407dbd227a897688b6f6ec273ef
Jul 29 18:45:58.961731 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.961644    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267975600"
Jul 29 18:45:58.961731 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.961690    2560 scope.go:117] "RemoveContainer" containerID="0c5868b71d82c84ad7fefc8225cf06e627178b60315d84c86d8d92a48e5a62c0"
Jul 29 18:45:58.962999 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:58.962802    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837267989296" event={"ID":"88446146-c743-4a14-94a1-6fb8d0de5d05","Type":"ContainerStarted","Data":"9c434125fa07a09ed06d3733b37e686ee7423407dbd227a897688b6f6ec273ef"}
Jul 29 18:45:59.971107 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:59.971057    2560 generic.go:356] "Generic (PLEG): container finished" podID="88446146-c743-4a14-94a1-6fb8d0de5d05" containerID="ce858329f72d5d4eeb5403189d50bf1719860ea4220f472da54cb76cf9958a20" exitCode=0
Jul 29 18:45:59.971622 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:45:59.971177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837267989296" event={"ID":"88446146-c743-4a14-94a1-6fb8d0de5d05","Type":"ContainerDied","Data":"ce858329f72d5d4eeb5403189d50bf1719860ea4220f472da54cb76cf9958a20"}
Jul 29 18:46:01.113971 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.113939    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:46:01.131741 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.131716    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7720-139837267989296_88446146-c743-4a14-94a1-6fb8d0de5d05/test-curl-7720-139837267989296/0.log"
Jul 29 18:46:01.154299 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.154245    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7720-139837267989296"]
Jul 29 18:46:01.157733 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.157709    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7720-139837267989296"]
Jul 29 18:46:01.160796 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.160777    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c95dn\" (UniqueName: \"kubernetes.io/projected/88446146-c743-4a14-94a1-6fb8d0de5d05-kube-api-access-c95dn\") pod \"88446146-c743-4a14-94a1-6fb8d0de5d05\" (UID: \"88446146-c743-4a14-94a1-6fb8d0de5d05\") "
Jul 29 18:46:01.162968 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.162949    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88446146-c743-4a14-94a1-6fb8d0de5d05-kube-api-access-c95dn" (OuterVolumeSpecName: "kube-api-access-c95dn") pod "88446146-c743-4a14-94a1-6fb8d0de5d05" (UID: "88446146-c743-4a14-94a1-6fb8d0de5d05"). InnerVolumeSpecName "kube-api-access-c95dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:46:01.262386 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.262354    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c95dn\" (UniqueName: \"kubernetes.io/projected/88446146-c743-4a14-94a1-6fb8d0de5d05-kube-api-access-c95dn\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:46:01.311358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.311323    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:01.311753 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.311739    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88446146-c743-4a14-94a1-6fb8d0de5d05" containerName="test-curl-7720-139837267989296"
Jul 29 18:46:01.311794 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.311755    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="88446146-c743-4a14-94a1-6fb8d0de5d05" containerName="test-curl-7720-139837267989296"
Jul 29 18:46:01.311832 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.311827    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="88446146-c743-4a14-94a1-6fb8d0de5d05" containerName="test-curl-7720-139837267989296"
Jul 29 18:46:01.313927 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.313913    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:01.321698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.321592    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:01.362916 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.362880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp85b\" (UniqueName: \"kubernetes.io/projected/eb9b1218-83d8-4806-bd93-e7fbf1202684-kube-api-access-wp85b\") pod \"test-curl-7720-139837268060608\" (UID: \"eb9b1218-83d8-4806-bd93-e7fbf1202684\") " pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:01.464423 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.464383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp85b\" (UniqueName: \"kubernetes.io/projected/eb9b1218-83d8-4806-bd93-e7fbf1202684-kube-api-access-wp85b\") pod \"test-curl-7720-139837268060608\" (UID: \"eb9b1218-83d8-4806-bd93-e7fbf1202684\") " pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:01.472325 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.472292    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp85b\" (UniqueName: \"kubernetes.io/projected/eb9b1218-83d8-4806-bd93-e7fbf1202684-kube-api-access-wp85b\") pod \"test-curl-7720-139837268060608\" (UID: \"eb9b1218-83d8-4806-bd93-e7fbf1202684\") " pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:01.625469 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.625366    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:01.754850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.754825    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:01.757317 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:46:01.757291    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb9b1218_83d8_4806_bd93_e7fbf1202684.slice/crio-2cd71242e1c892df028b6b3d38a4d8e0b97cb63912ae1a7ebfc405c858a5c75c WatchSource:0}: Error finding container 2cd71242e1c892df028b6b3d38a4d8e0b97cb63912ae1a7ebfc405c858a5c75c: Status 404 returned error can't find the container with id 2cd71242e1c892df028b6b3d38a4d8e0b97cb63912ae1a7ebfc405c858a5c75c
Jul 29 18:46:01.984240 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.984145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837268060608" event={"ID":"eb9b1218-83d8-4806-bd93-e7fbf1202684","Type":"ContainerStarted","Data":"2cd71242e1c892df028b6b3d38a4d8e0b97cb63912ae1a7ebfc405c858a5c75c"}
Jul 29 18:46:01.985527 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.985499    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c434125fa07a09ed06d3733b37e686ee7423407dbd227a897688b6f6ec273ef"
Jul 29 18:46:01.985680 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:01.985528    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:46:01.995281 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:46:01.995241    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7720-139837267989296\" is forbidden: User \"system:node:ip-10-0-130-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-240.ec2.internal' and this object" podUID="88446146-c743-4a14-94a1-6fb8d0de5d05" pod="opendatahub/test-curl-7720-139837267989296"
Jul 29 18:46:02.282000 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:02.281965    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88446146-c743-4a14-94a1-6fb8d0de5d05" path="/var/lib/kubelet/pods/88446146-c743-4a14-94a1-6fb8d0de5d05/volumes"
Jul 29 18:46:02.991447 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:02.991410    2560 generic.go:356] "Generic (PLEG): container finished" podID="eb9b1218-83d8-4806-bd93-e7fbf1202684" containerID="0272e39ba5ad43becfc45f1f721083100a601bb14e64f7341bf3b1cf81bd3623" exitCode=0
Jul 29 18:46:02.991617 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:02.991453    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837268060608" event={"ID":"eb9b1218-83d8-4806-bd93-e7fbf1202684","Type":"ContainerDied","Data":"0272e39ba5ad43becfc45f1f721083100a601bb14e64f7341bf3b1cf81bd3623"}
Jul 29 18:46:04.132075 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.132045    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:04.152066 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.152038    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7720-139837268060608_eb9b1218-83d8-4806-bd93-e7fbf1202684/test-curl-7720-139837268060608/0.log"
Jul 29 18:46:04.171557 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.171527    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:04.175108 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.175081    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:04.190152 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.190126    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp85b\" (UniqueName: \"kubernetes.io/projected/eb9b1218-83d8-4806-bd93-e7fbf1202684-kube-api-access-wp85b\") pod \"eb9b1218-83d8-4806-bd93-e7fbf1202684\" (UID: \"eb9b1218-83d8-4806-bd93-e7fbf1202684\") "
Jul 29 18:46:04.192286 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.192241    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9b1218-83d8-4806-bd93-e7fbf1202684-kube-api-access-wp85b" (OuterVolumeSpecName: "kube-api-access-wp85b") pod "eb9b1218-83d8-4806-bd93-e7fbf1202684" (UID: "eb9b1218-83d8-4806-bd93-e7fbf1202684"). InnerVolumeSpecName "kube-api-access-wp85b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:46:04.281799 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.281706    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb9b1218-83d8-4806-bd93-e7fbf1202684" path="/var/lib/kubelet/pods/eb9b1218-83d8-4806-bd93-e7fbf1202684/volumes"
Jul 29 18:46:04.291829 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.291801    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp85b\" (UniqueName: \"kubernetes.io/projected/eb9b1218-83d8-4806-bd93-e7fbf1202684-kube-api-access-wp85b\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:46:04.292570 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.292551    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:04.292967 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.292954    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb9b1218-83d8-4806-bd93-e7fbf1202684" containerName="test-curl-7720-139837268060608"
Jul 29 18:46:04.293004 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.292969    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb9b1218-83d8-4806-bd93-e7fbf1202684" containerName="test-curl-7720-139837268060608"
Jul 29 18:46:04.293043 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.293029    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb9b1218-83d8-4806-bd93-e7fbf1202684" containerName="test-curl-7720-139837268060608"
Jul 29 18:46:04.295179 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.295161    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:04.299640 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.299617    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:04.392764 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.392729    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2lsj\" (UniqueName: \"kubernetes.io/projected/10748181-ef1d-4211-bb8c-58f6d959f84f-kube-api-access-s2lsj\") pod \"test-curl-7720-139837268060608\" (UID: \"10748181-ef1d-4211-bb8c-58f6d959f84f\") " pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:04.494345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.494312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2lsj\" (UniqueName: \"kubernetes.io/projected/10748181-ef1d-4211-bb8c-58f6d959f84f-kube-api-access-s2lsj\") pod \"test-curl-7720-139837268060608\" (UID: \"10748181-ef1d-4211-bb8c-58f6d959f84f\") " pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:04.501714 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.501684    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2lsj\" (UniqueName: \"kubernetes.io/projected/10748181-ef1d-4211-bb8c-58f6d959f84f-kube-api-access-s2lsj\") pod \"test-curl-7720-139837268060608\" (UID: \"10748181-ef1d-4211-bb8c-58f6d959f84f\") " pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:04.606336 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.606231    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:04.737593 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:04.737567    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:04.739778 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:46:04.739748    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10748181_ef1d_4211_bb8c_58f6d959f84f.slice/crio-9cec11e113a0f85e5d284f8e6d6d19d1344b149bbe8ff222a7c7c7f893d9e5bb WatchSource:0}: Error finding container 9cec11e113a0f85e5d284f8e6d6d19d1344b149bbe8ff222a7c7c7f893d9e5bb: Status 404 returned error can't find the container with id 9cec11e113a0f85e5d284f8e6d6d19d1344b149bbe8ff222a7c7c7f893d9e5bb
Jul 29 18:46:05.003312 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:05.003246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837268060608" event={"ID":"10748181-ef1d-4211-bb8c-58f6d959f84f","Type":"ContainerStarted","Data":"9cec11e113a0f85e5d284f8e6d6d19d1344b149bbe8ff222a7c7c7f893d9e5bb"}
Jul 29 18:46:05.004446 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:05.004428    2560 scope.go:117] "RemoveContainer" containerID="0272e39ba5ad43becfc45f1f721083100a601bb14e64f7341bf3b1cf81bd3623"
Jul 29 18:46:05.004446 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:05.004437    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:06.012816 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:06.012784    2560 generic.go:356] "Generic (PLEG): container finished" podID="10748181-ef1d-4211-bb8c-58f6d959f84f" containerID="6a755ca77719ec946b5a7634059aaf7c7c55f3dd5cf681f7f7718c1eb67fb727" exitCode=0
Jul 29 18:46:06.013316 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:06.012834    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837268060608" event={"ID":"10748181-ef1d-4211-bb8c-58f6d959f84f","Type":"ContainerDied","Data":"6a755ca77719ec946b5a7634059aaf7c7c55f3dd5cf681f7f7718c1eb67fb727"}
Jul 29 18:46:07.154215 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.154189    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:07.173841 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.173812    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7720-139837268060608_10748181-ef1d-4211-bb8c-58f6d959f84f/test-curl-7720-139837268060608/0.log"
Jul 29 18:46:07.194956 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.194919    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:07.198921 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.198893    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7720-139837268060608"]
Jul 29 18:46:07.218954 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.218922    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2lsj\" (UniqueName: \"kubernetes.io/projected/10748181-ef1d-4211-bb8c-58f6d959f84f-kube-api-access-s2lsj\") pod \"10748181-ef1d-4211-bb8c-58f6d959f84f\" (UID: \"10748181-ef1d-4211-bb8c-58f6d959f84f\") "
Jul 29 18:46:07.221211 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.221186    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10748181-ef1d-4211-bb8c-58f6d959f84f-kube-api-access-s2lsj" (OuterVolumeSpecName: "kube-api-access-s2lsj") pod "10748181-ef1d-4211-bb8c-58f6d959f84f" (UID: "10748181-ef1d-4211-bb8c-58f6d959f84f"). InnerVolumeSpecName "kube-api-access-s2lsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:46:07.313430 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.313344    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7720-139837267673568"]
Jul 29 18:46:07.313783 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.313770    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10748181-ef1d-4211-bb8c-58f6d959f84f" containerName="test-curl-7720-139837268060608"
Jul 29 18:46:07.313836 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.313785    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="10748181-ef1d-4211-bb8c-58f6d959f84f" containerName="test-curl-7720-139837268060608"
Jul 29 18:46:07.313873 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.313864    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="10748181-ef1d-4211-bb8c-58f6d959f84f" containerName="test-curl-7720-139837268060608"
Jul 29 18:46:07.316119 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.316101    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267673568"
Jul 29 18:46:07.319897 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.319862    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2lsj\" (UniqueName: \"kubernetes.io/projected/10748181-ef1d-4211-bb8c-58f6d959f84f-kube-api-access-s2lsj\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:46:07.323343 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.323319    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837267673568"]
Jul 29 18:46:07.420915 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.420881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jncp4\" (UniqueName: \"kubernetes.io/projected/14a4d2fb-121b-418c-9c3a-8200affc37c4-kube-api-access-jncp4\") pod \"test-curl-7720-139837267673568\" (UID: \"14a4d2fb-121b-418c-9c3a-8200affc37c4\") " pod="opendatahub/test-curl-7720-139837267673568"
Jul 29 18:46:07.522359 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.522307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jncp4\" (UniqueName: \"kubernetes.io/projected/14a4d2fb-121b-418c-9c3a-8200affc37c4-kube-api-access-jncp4\") pod \"test-curl-7720-139837267673568\" (UID: \"14a4d2fb-121b-418c-9c3a-8200affc37c4\") " pod="opendatahub/test-curl-7720-139837267673568"
Jul 29 18:46:07.529873 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.529844    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jncp4\" (UniqueName: \"kubernetes.io/projected/14a4d2fb-121b-418c-9c3a-8200affc37c4-kube-api-access-jncp4\") pod \"test-curl-7720-139837267673568\" (UID: \"14a4d2fb-121b-418c-9c3a-8200affc37c4\") " pod="opendatahub/test-curl-7720-139837267673568"
Jul 29 18:46:07.627468 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.627349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267673568"
Jul 29 18:46:07.760047 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:07.760000    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-139837267673568"]
Jul 29 18:46:07.768082 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:46:07.768056    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14a4d2fb_121b_418c_9c3a_8200affc37c4.slice/crio-18c291e93f1bdf35bca707344bdcee95a3f6defa69ad8da4257bf797fdc1d4b7 WatchSource:0}: Error finding container 18c291e93f1bdf35bca707344bdcee95a3f6defa69ad8da4257bf797fdc1d4b7: Status 404 returned error can't find the container with id 18c291e93f1bdf35bca707344bdcee95a3f6defa69ad8da4257bf797fdc1d4b7
Jul 29 18:46:08.024249 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:08.024214    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:08.024249 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:08.024239    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cec11e113a0f85e5d284f8e6d6d19d1344b149bbe8ff222a7c7c7f893d9e5bb"
Jul 29 18:46:08.025622 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:08.025587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837267673568" event={"ID":"14a4d2fb-121b-418c-9c3a-8200affc37c4","Type":"ContainerStarted","Data":"18c291e93f1bdf35bca707344bdcee95a3f6defa69ad8da4257bf797fdc1d4b7"}
Jul 29 18:46:08.034152 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:46:08.034127    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7720-139837268060608\" is forbidden: User \"system:node:ip-10-0-130-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-240.ec2.internal' and this object" podUID="10748181-ef1d-4211-bb8c-58f6d959f84f" pod="opendatahub/test-curl-7720-139837268060608"
Jul 29 18:46:08.285720 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:08.285638    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10748181-ef1d-4211-bb8c-58f6d959f84f" path="/var/lib/kubelet/pods/10748181-ef1d-4211-bb8c-58f6d959f84f/volumes"
Jul 29 18:46:09.032751 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:09.032721    2560 generic.go:356] "Generic (PLEG): container finished" podID="14a4d2fb-121b-418c-9c3a-8200affc37c4" containerID="a030702748e7e7e92cd2536a5da3ec7657c8f4aa91cf6807e92302a5879b751f" exitCode=0
Jul 29 18:46:09.032925 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:09.032777    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-139837267673568" event={"ID":"14a4d2fb-121b-418c-9c3a-8200affc37c4","Type":"ContainerDied","Data":"a030702748e7e7e92cd2536a5da3ec7657c8f4aa91cf6807e92302a5879b751f"}
Jul 29 18:46:10.178103 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.178077    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267673568"
Jul 29 18:46:10.194813 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.194789    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7720-139837267673568_14a4d2fb-121b-418c-9c3a-8200affc37c4/test-curl-7720-139837267673568/0.log"
Jul 29 18:46:10.214979 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.214945    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7720-139837267673568"]
Jul 29 18:46:10.217954 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.217926    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7720-139837267673568"]
Jul 29 18:46:10.249052 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.249024    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jncp4\" (UniqueName: \"kubernetes.io/projected/14a4d2fb-121b-418c-9c3a-8200affc37c4-kube-api-access-jncp4\") pod \"14a4d2fb-121b-418c-9c3a-8200affc37c4\" (UID: \"14a4d2fb-121b-418c-9c3a-8200affc37c4\") "
Jul 29 18:46:10.251300 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.251246    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a4d2fb-121b-418c-9c3a-8200affc37c4-kube-api-access-jncp4" (OuterVolumeSpecName: "kube-api-access-jncp4") pod "14a4d2fb-121b-418c-9c3a-8200affc37c4" (UID: "14a4d2fb-121b-418c-9c3a-8200affc37c4"). InnerVolumeSpecName "kube-api-access-jncp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:46:10.282956 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.282912    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14a4d2fb-121b-418c-9c3a-8200affc37c4" path="/var/lib/kubelet/pods/14a4d2fb-121b-418c-9c3a-8200affc37c4/volumes"
Jul 29 18:46:10.350804 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:10.350772    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jncp4\" (UniqueName: \"kubernetes.io/projected/14a4d2fb-121b-418c-9c3a-8200affc37c4-kube-api-access-jncp4\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:46:11.044908 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:11.044882    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-139837267673568"
Jul 29 18:46:11.045093 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:11.044881    2560 scope.go:117] "RemoveContainer" containerID="a030702748e7e7e92cd2536a5da3ec7657c8f4aa91cf6807e92302a5879b751f"
Jul 29 18:46:14.207312 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.207254    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"]
Jul 29 18:46:14.207906 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.207886    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14a4d2fb-121b-418c-9c3a-8200affc37c4" containerName="test-curl-7720-139837267673568"
Jul 29 18:46:14.208003 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.207910    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="14a4d2fb-121b-418c-9c3a-8200affc37c4" containerName="test-curl-7720-139837267673568"
Jul 29 18:46:14.208058 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.208015    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="14a4d2fb-121b-418c-9c3a-8200affc37c4" containerName="test-curl-7720-139837267673568"
Jul 29 18:46:14.211066 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.211042    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.212957 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.212938    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-479e5c-openshift-default-dockercfg-f7hfs\""
Jul 29 18:46:14.224883 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.224861    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"]
Jul 29 18:46:14.292588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-envoy\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.292769 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292598    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istiod-ca-cert\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.292769 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-certs\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.292769 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-socket\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.292769 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292746    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-podinfo\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.292910 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292870    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-token\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.292948 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292907    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4h22\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-kube-api-access-m4h22\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.292948 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292930    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-credential-socket\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.293024 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.292957    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-data\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393568 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393523    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-token\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393575    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4h22\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-kube-api-access-m4h22\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-credential-socket\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393772 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-data\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393793    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-envoy\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393846    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istiod-ca-cert\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393876    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-certs\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-socket\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.393963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.393928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-podinfo\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.394200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.394050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-data\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.394200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.394093    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-credential-socket\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.394351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.394248    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-socket\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.394351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.394334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-certs\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.394478 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.394460    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istiod-ca-cert\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.396328 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.396296    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-envoy\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.396507 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.396486    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-podinfo\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.401315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.401291    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-token\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.401575 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.401558    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4h22\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-kube-api-access-m4h22\") pod \"e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") " pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.521907 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.521862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:14.663315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.663290    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"]
Jul 29 18:46:14.665072 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:46:14.665043    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed3be72_fbdf_4561_a5b7_7dab60583c67.slice/crio-5ee0ce0b91b99c9ac11e62c2cfc8e0128ca3ebf7244f19c6b47b23bc1475282f WatchSource:0}: Error finding container 5ee0ce0b91b99c9ac11e62c2cfc8e0128ca3ebf7244f19c6b47b23bc1475282f: Status 404 returned error can't find the container with id 5ee0ce0b91b99c9ac11e62c2cfc8e0128ca3ebf7244f19c6b47b23bc1475282f
Jul 29 18:46:14.667235 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.667203    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:46:14.667349 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.667289    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:46:14.667349 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:14.667321    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:46:15.068253 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:15.068213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" event={"ID":"1ed3be72-fbdf-4561-a5b7-7dab60583c67","Type":"ContainerStarted","Data":"ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3"}
Jul 29 18:46:15.068430 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:15.068287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" event={"ID":"1ed3be72-fbdf-4561-a5b7-7dab60583c67","Type":"ContainerStarted","Data":"5ee0ce0b91b99c9ac11e62c2cfc8e0128ca3ebf7244f19c6b47b23bc1475282f"}
Jul 29 18:46:15.087551 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:15.087498    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podStartSLOduration=1.08748196 podStartE2EDuration="1.08748196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:46:15.085933808 +0000 UTC m=+3165.327894847" watchObservedRunningTime="2026-07-29 18:46:15.08748196 +0000 UTC m=+3165.329442999"
Jul 29 18:46:15.522926 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:15.522881    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:15.524299 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:15.524260    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 29 18:46:15.524414 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:15.524328    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 29 18:46:16.526578 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:16.526545    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:17.079965 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:17.079928    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:17.080914 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:17.080893    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:18.054555 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.054521    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"]
Jul 29 18:46:18.058282 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.058250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.060359 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.060337    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-479e5c-serving-cert\""
Jul 29 18:46:18.069441 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.069417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"]
Jul 29 18:46:18.134583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.134545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.134770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.134675    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.134770 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.134706    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ql2c\" (UniqueName: \"kubernetes.io/projected/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-kube-api-access-7ql2c\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.237306 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.236461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.237306 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.236515    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ql2c\" (UniqueName: \"kubernetes.io/projected/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-kube-api-access-7ql2c\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.237306 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.236654    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.237611 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.237521    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.240671 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.240640    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.247798 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.247766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ql2c\" (UniqueName: \"kubernetes.io/projected/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-kube-api-access-7ql2c\") pod \"maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.372316 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.372203    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:18.519403 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:18.519373    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"]
Jul 29 18:46:18.521142 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:46:18.521108    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554e07b2_c7dc_4deb_8227_e9442d4fdc9e.slice/crio-e80d2bb9e57f226fac54f5b3c2f90b8e835d9f0725a6b86f07ad98562125cf85 WatchSource:0}: Error finding container e80d2bb9e57f226fac54f5b3c2f90b8e835d9f0725a6b86f07ad98562125cf85: Status 404 returned error can't find the container with id e80d2bb9e57f226fac54f5b3c2f90b8e835d9f0725a6b86f07ad98562125cf85
Jul 29 18:46:19.092278 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:19.092219    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw" event={"ID":"554e07b2-c7dc-4deb-8227-e9442d4fdc9e","Type":"ContainerStarted","Data":"29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2"}
Jul 29 18:46:19.092774 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:19.092288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw" event={"ID":"554e07b2-c7dc-4deb-8227-e9442d4fdc9e","Type":"ContainerStarted","Data":"e80d2bb9e57f226fac54f5b3c2f90b8e835d9f0725a6b86f07ad98562125cf85"}
Jul 29 18:46:19.092774 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:19.092323    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:19.125151 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:19.125088    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw" podStartSLOduration=0.67774733 podStartE2EDuration="1.125070408s" totalImagesPullingTime="447.323078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:46:19.123390109 +0000 UTC m=+3169.365351170" watchObservedRunningTime="2026-07-29 18:46:19.125070408 +0000 UTC m=+3169.367031475"
Jul 29 18:46:25.103483 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:25.103451    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:46:33.123455 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:33.123408    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 18:46:33.123862 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:33.123484    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 18:46:35.179525 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:35.179433    2560 generic.go:356] "Generic (PLEG): container finished" podID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerID="ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3" exitCode=0
Jul 29 18:46:35.180025 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:35.179513    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" event={"ID":"1ed3be72-fbdf-4561-a5b7-7dab60583c67","Type":"ContainerDied","Data":"ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3"}
Jul 29 18:46:35.180090 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:35.180062    2560 scope.go:117] "RemoveContainer" containerID="ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3"
Jul 29 18:46:35.181225 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:35.181198    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:46:35.181352 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:35.181306    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:46:35.181427 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:35.181356    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:46:36.186043 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:36.186009    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" event={"ID":"1ed3be72-fbdf-4561-a5b7-7dab60583c67","Type":"ContainerStarted","Data":"18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4"}
Jul 29 18:46:36.523130 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:36.523082    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:36.523387 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:36.523229    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 29 18:46:36.523387 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:36.523300    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 29 18:46:37.522731 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:37.522296    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 29 18:46:37.522731 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:37.522357    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 29 18:46:38.522452 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:38.522412    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 29 18:46:38.522775 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:38.522738    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 29 18:46:39.523247 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:39.523206    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 29 18:46:39.523902 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:39.523841    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 29 18:46:40.525783 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:40.525747    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:40.526158 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:40.526091    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:40.526757 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:40.526739    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:46:46.486762 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.486728    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"]
Jul 29 18:46:46.490406 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.490372    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.492361 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.492341    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-b0fdf2-serving-cert\""
Jul 29 18:46:46.501345 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.501314    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"]
Jul 29 18:46:46.636388 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.636347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e419db-a9f8-496e-935d-530e4fbcef51-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.636584 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.636514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp2m9\" (UniqueName: \"kubernetes.io/projected/14e419db-a9f8-496e-935d-530e4fbcef51-kube-api-access-sp2m9\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.636584 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.636555    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e419db-a9f8-496e-935d-530e4fbcef51-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.738172 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.738079    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp2m9\" (UniqueName: \"kubernetes.io/projected/14e419db-a9f8-496e-935d-530e4fbcef51-kube-api-access-sp2m9\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.738172 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.738124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e419db-a9f8-496e-935d-530e4fbcef51-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.738444 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.738190    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e419db-a9f8-496e-935d-530e4fbcef51-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.738960 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.738894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e419db-a9f8-496e-935d-530e4fbcef51-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.740785 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.740763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e419db-a9f8-496e-935d-530e4fbcef51-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.756037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.756010    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp2m9\" (UniqueName: \"kubernetes.io/projected/14e419db-a9f8-496e-935d-530e4fbcef51-kube-api-access-sp2m9\") pod \"maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.803971 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.803934    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:46.947868 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:46.947839    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"]
Jul 29 18:46:46.950149 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:46:46.950124    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e419db_a9f8_496e_935d_530e4fbcef51.slice/crio-5c738f5f6e958bf8676c95843a48ef5d2f2e5daf681d58c63935828c1b65ce27 WatchSource:0}: Error finding container 5c738f5f6e958bf8676c95843a48ef5d2f2e5daf681d58c63935828c1b65ce27: Status 404 returned error can't find the container with id 5c738f5f6e958bf8676c95843a48ef5d2f2e5daf681d58c63935828c1b65ce27
Jul 29 18:46:47.248337 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:47.248302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5" event={"ID":"14e419db-a9f8-496e-935d-530e4fbcef51","Type":"ContainerStarted","Data":"5c738f5f6e958bf8676c95843a48ef5d2f2e5daf681d58c63935828c1b65ce27"}
Jul 29 18:46:48.255525 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:48.255493    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5" event={"ID":"14e419db-a9f8-496e-935d-530e4fbcef51","Type":"ContainerStarted","Data":"364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71"}
Jul 29 18:46:48.255912 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:48.255599    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:46:48.271419 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:48.271371    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5" podStartSLOduration=1.827190696 podStartE2EDuration="2.271354667s" totalImagesPullingTime="444.163971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:46:48.269655931 +0000 UTC m=+3198.511616969" watchObservedRunningTime="2026-07-29 18:46:48.271354667 +0000 UTC m=+3198.513315741"
Jul 29 18:46:54.266851 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:46:54.266826    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:48:20.221915 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.221881    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"]
Jul 29 18:48:20.225533 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.225511    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.227550 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.227531    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-8b8f04-openshift-default-dockercfg-thlfm\""
Jul 29 18:48:20.240516 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.240490    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"]
Jul 29 18:48:20.271164 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-credential-socket\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271191    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-podinfo\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271218    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-socket\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-data\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271374    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-certs\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-envoy\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271529    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-token\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.271730 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.271609    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvkxj\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-kube-api-access-nvkxj\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.372594 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372554    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-token\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.372799 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvkxj\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-kube-api-access-nvkxj\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.372799 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372665    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.372799 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-credential-socket\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.372799 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372727    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-podinfo\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.372799 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-socket\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372809    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-data\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372837    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-certs\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373077 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.372862    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-envoy\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373370 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.373339    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-credential-socket\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373501 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.373382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-certs\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373501 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.373418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-socket\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373501 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.373448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.373634 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.373506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-data\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.375501 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.375480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-envoy\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.375614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.375512    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-podinfo\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.381915 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.381894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-token\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.383922 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.383903    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvkxj\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-kube-api-access-nvkxj\") pod \"e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") " pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.536060 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.536025    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:20.683178 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.683152    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"]
Jul 29 18:48:20.684748 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:48:20.684709    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fe7b45_d52c_4081_9d57_51c2f2deebbe.slice/crio-9f58fdaeffd02a52c3cacb38b426e15cd3585f486a3f1843dc293a7b51da0018 WatchSource:0}: Error finding container 9f58fdaeffd02a52c3cacb38b426e15cd3585f486a3f1843dc293a7b51da0018: Status 404 returned error can't find the container with id 9f58fdaeffd02a52c3cacb38b426e15cd3585f486a3f1843dc293a7b51da0018
Jul 29 18:48:20.686843 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.686813    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:48:20.686950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.686875    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:48:20.686950 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.686917    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:48:20.750627 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:20.750594    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" event={"ID":"c6fe7b45-d52c-4081-9d57-51c2f2deebbe","Type":"ContainerStarted","Data":"9f58fdaeffd02a52c3cacb38b426e15cd3585f486a3f1843dc293a7b51da0018"}
Jul 29 18:48:21.758093 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:21.758050    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" event={"ID":"c6fe7b45-d52c-4081-9d57-51c2f2deebbe","Type":"ContainerStarted","Data":"f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7"}
Jul 29 18:48:21.780901 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:21.780842    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" podStartSLOduration=1.780826191 podStartE2EDuration="1.780826191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:48:21.778347978 +0000 UTC m=+3292.020309017" watchObservedRunningTime="2026-07-29 18:48:21.780826191 +0000 UTC m=+3292.022787229"
Jul 29 18:48:22.536789 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:22.536752    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:22.541743 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:22.541717    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:22.763217 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:22.763175    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:22.764196 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:22.764173    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:24.783714 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:24.783682    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"]
Jul 29 18:48:24.788258 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:24.788231    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:24.790346 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:24.790323    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-8b8f04-serving-cert\""
Jul 29 18:48:24.801824 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:24.801800    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"]
Jul 29 18:48:24.924557 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:24.924527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:24.924740 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:24.924572    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf8pp\" (UniqueName: \"kubernetes.io/projected/0bffc5bc-47ca-4a71-a635-62abbe723b08-kube-api-access-rf8pp\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:24.924740 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:24.924595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bffc5bc-47ca-4a71-a635-62abbe723b08-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.026235 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.026195    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.026429 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.026248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf8pp\" (UniqueName: \"kubernetes.io/projected/0bffc5bc-47ca-4a71-a635-62abbe723b08-kube-api-access-rf8pp\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.026429 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.026304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bffc5bc-47ca-4a71-a635-62abbe723b08-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.026429 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:48:25.026396    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-serving-cert: secret "maas-api-e2e-ipp-cleanup-8b8f04-serving-cert" not found
Jul 29 18:48:25.026546 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:48:25.026484    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls podName:0bffc5bc-47ca-4a71-a635-62abbe723b08 nodeName:}" failed. No retries permitted until 2026-07-29 18:48:25.526462457 +0000 UTC m=+3295.768423473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz" (UID: "0bffc5bc-47ca-4a71-a635-62abbe723b08") : secret "maas-api-e2e-ipp-cleanup-8b8f04-serving-cert" not found
Jul 29 18:48:25.026992 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.026974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bffc5bc-47ca-4a71-a635-62abbe723b08-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.034804 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.034742    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf8pp\" (UniqueName: \"kubernetes.io/projected/0bffc5bc-47ca-4a71-a635-62abbe723b08-kube-api-access-rf8pp\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.531542 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.531506    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.534006 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.533985    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.701740 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.701706    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:25.850117 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:25.850087    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"]
Jul 29 18:48:26.788130 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:26.788092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz" event={"ID":"0bffc5bc-47ca-4a71-a635-62abbe723b08","Type":"ContainerStarted","Data":"8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c"}
Jul 29 18:48:26.788338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:26.788140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz" event={"ID":"0bffc5bc-47ca-4a71-a635-62abbe723b08","Type":"ContainerStarted","Data":"caa99f7fd86bbcfe6221e75d46cbefd2ae96fba79961b8957f21a9b35ab2969d"}
Jul 29 18:48:26.788338 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:26.788170    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:26.810013 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:26.809483    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz" podStartSLOduration=2.334681908 podStartE2EDuration="2.809463301s" totalImagesPullingTime="474.781393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:48:26.803283748 +0000 UTC m=+3297.045244780" watchObservedRunningTime="2026-07-29 18:48:26.809463301 +0000 UTC m=+3297.051424340"
Jul 29 18:48:30.467478 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:30.467454    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:48:30.474786 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:30.474767    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xw7vr_8e5e8cfb-8273-44f3-ac0c-fce49b77c268/ovn-acl-logging/0.log"
Jul 29 18:48:32.799254 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:32.799219    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:38.804447 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:38.804402    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 18:48:38.804844 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:38.804494    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 18:48:39.870570 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:39.870538    2560 generic.go:356] "Generic (PLEG): container finished" podID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerID="f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7" exitCode=0
Jul 29 18:48:39.870977 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:39.870615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" event={"ID":"c6fe7b45-d52c-4081-9d57-51c2f2deebbe","Type":"ContainerDied","Data":"f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7"}
Jul 29 18:48:39.871053 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:39.871038    2560 scope.go:117] "RemoveContainer" containerID="f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7"
Jul 29 18:48:39.872132 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:39.872102    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:48:39.872188 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:39.872173    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:48:39.872225 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:39.872204    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 18:48:40.877692 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:40.877655    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" event={"ID":"c6fe7b45-d52c-4081-9d57-51c2f2deebbe","Type":"ContainerStarted","Data":"11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a"}
Jul 29 18:48:41.536787 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:41.536742    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:41.537021 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:41.536990    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 29 18:48:41.537107 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:41.537051    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 29 18:48:42.536921 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:42.536885    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 29 18:48:42.537330 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:42.536943    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 29 18:48:43.537180 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:43.537144    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 29 18:48:43.537586 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:43.537201    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 29 18:48:44.537499 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.537455    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 29 18:48:44.537920 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.537547    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 29 18:48:44.585601 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.585560    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"]
Jul 29 18:48:44.585867 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.585820    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz" podUID="0bffc5bc-47ca-4a71-a635-62abbe723b08" containerName="maas-api" containerID="cri-o://8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c" gracePeriod=30
Jul 29 18:48:44.903652 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.903623    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:44.905733 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.905704    2560 generic.go:356] "Generic (PLEG): container finished" podID="0bffc5bc-47ca-4a71-a635-62abbe723b08" containerID="8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c" exitCode=0
Jul 29 18:48:44.905854 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.905749    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"
Jul 29 18:48:44.905854 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.905788    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz" event={"ID":"0bffc5bc-47ca-4a71-a635-62abbe723b08","Type":"ContainerDied","Data":"8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c"}
Jul 29 18:48:44.905854 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.905849    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz" event={"ID":"0bffc5bc-47ca-4a71-a635-62abbe723b08","Type":"ContainerDied","Data":"caa99f7fd86bbcfe6221e75d46cbefd2ae96fba79961b8957f21a9b35ab2969d"}
Jul 29 18:48:44.906036 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.905911    2560 scope.go:117] "RemoveContainer" containerID="8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c"
Jul 29 18:48:44.928300 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.928202    2560 scope.go:117] "RemoveContainer" containerID="8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c"
Jul 29 18:48:44.928831 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:48:44.928801    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c\": container with ID starting with 8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c not found: ID does not exist" containerID="8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c"
Jul 29 18:48:44.928936 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.928843    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c"} err="failed to get container status \"8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c\": rpc error: code = NotFound desc = could not find container \"8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c\": container with ID starting with 8ed7f146389e161f713a480eb95ee22e26c65881e3949872831c06de2edc4f3c not found: ID does not exist"
Jul 29 18:48:44.969647 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.969613    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf8pp\" (UniqueName: \"kubernetes.io/projected/0bffc5bc-47ca-4a71-a635-62abbe723b08-kube-api-access-rf8pp\") pod \"0bffc5bc-47ca-4a71-a635-62abbe723b08\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") "
Jul 29 18:48:44.969647 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.969652    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bffc5bc-47ca-4a71-a635-62abbe723b08-openshift-service-ca\") pod \"0bffc5bc-47ca-4a71-a635-62abbe723b08\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") "
Jul 29 18:48:44.969932 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.969803    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls\") pod \"0bffc5bc-47ca-4a71-a635-62abbe723b08\" (UID: \"0bffc5bc-47ca-4a71-a635-62abbe723b08\") "
Jul 29 18:48:44.970081 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.970043    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bffc5bc-47ca-4a71-a635-62abbe723b08-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0bffc5bc-47ca-4a71-a635-62abbe723b08" (UID: "0bffc5bc-47ca-4a71-a635-62abbe723b08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:48:44.972052 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.972023    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0bffc5bc-47ca-4a71-a635-62abbe723b08" (UID: "0bffc5bc-47ca-4a71-a635-62abbe723b08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:48:44.972148 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:44.972053    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bffc5bc-47ca-4a71-a635-62abbe723b08-kube-api-access-rf8pp" (OuterVolumeSpecName: "kube-api-access-rf8pp") pod "0bffc5bc-47ca-4a71-a635-62abbe723b08" (UID: "0bffc5bc-47ca-4a71-a635-62abbe723b08"). InnerVolumeSpecName "kube-api-access-rf8pp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:48:45.070969 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.070939    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf8pp\" (UniqueName: \"kubernetes.io/projected/0bffc5bc-47ca-4a71-a635-62abbe723b08-kube-api-access-rf8pp\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:45.070969 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.070967    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bffc5bc-47ca-4a71-a635-62abbe723b08-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:45.071162 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.070977    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bffc5bc-47ca-4a71-a635-62abbe723b08-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:45.227676 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.227647    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"]
Jul 29 18:48:45.230827 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.230805    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8b8f04-846c9dcdc-p7qjz"]
Jul 29 18:48:45.540865 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.540836    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:45.541236 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.541119    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:45.541998 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:45.541979    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:46.282408 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:46.282378    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bffc5bc-47ca-4a71-a635-62abbe723b08" path="/var/lib/kubelet/pods/0bffc5bc-47ca-4a71-a635-62abbe723b08/volumes"
Jul 29 18:48:51.253184 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:51.253111    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"]
Jul 29 18:48:51.253623 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:51.253359    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy" containerID="cri-o://11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a" gracePeriod=30
Jul 29 18:48:56.516759 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.516692    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:56.584850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.584816    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-socket\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.584871    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istiod-ca-cert\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.584892    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-envoy\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.584908    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvkxj\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-kube-api-access-nvkxj\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.584930    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-podinfo\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.584968    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-credential-socket\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585016 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585003    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-token\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585064    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-data\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585116    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-certs\") pod \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\" (UID: \"c6fe7b45-d52c-4081-9d57-51c2f2deebbe\") "
Jul 29 18:48:56.585358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585190    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:48:56.585358 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585339    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:48:56.585583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585462    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:48:56.585583 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585478    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:48:56.585693 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585654    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.585693 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585676    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.586037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585695    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.586037 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.585713    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.586364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.586339    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:48:56.587304 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.587253    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-kube-api-access-nvkxj" (OuterVolumeSpecName: "kube-api-access-nvkxj") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "kube-api-access-nvkxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:48:56.587395 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.587336    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:48:56.587395 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.587351    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:48:56.587475 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.587430    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6fe7b45-d52c-4081-9d57-51c2f2deebbe" (UID: "c6fe7b45-d52c-4081-9d57-51c2f2deebbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:48:56.686855 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.686820    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.686855 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.686846    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvkxj\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-kube-api-access-nvkxj\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.686855 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.686859    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.687080 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.686873    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.687080 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.686882    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6fe7b45-d52c-4081-9d57-51c2f2deebbe-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:48:56.975631 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.975540    2560 generic.go:356] "Generic (PLEG): container finished" podID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerID="11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a" exitCode=0
Jul 29 18:48:56.975631 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.975604    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"
Jul 29 18:48:56.975830 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.975620    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" event={"ID":"c6fe7b45-d52c-4081-9d57-51c2f2deebbe","Type":"ContainerDied","Data":"11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a"}
Jul 29 18:48:56.975830 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.975679    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln" event={"ID":"c6fe7b45-d52c-4081-9d57-51c2f2deebbe","Type":"ContainerDied","Data":"9f58fdaeffd02a52c3cacb38b426e15cd3585f486a3f1843dc293a7b51da0018"}
Jul 29 18:48:56.975830 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.975701    2560 scope.go:117] "RemoveContainer" containerID="11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a"
Jul 29 18:48:56.985584 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.985566    2560 scope.go:117] "RemoveContainer" containerID="f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7"
Jul 29 18:48:56.994350 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.994306    2560 scope.go:117] "RemoveContainer" containerID="11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a"
Jul 29 18:48:56.994774 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:48:56.994753    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a\": container with ID starting with 11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a not found: ID does not exist" containerID="11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a"
Jul 29 18:48:56.994848 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.994783    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a"} err="failed to get container status \"11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a\": rpc error: code = NotFound desc = could not find container \"11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a\": container with ID starting with 11cb4c25bb7b6e2d5da65be3e2ac1ec8497acf2d410a3ab0b4f08735be45d50a not found: ID does not exist"
Jul 29 18:48:56.994848 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.994802    2560 scope.go:117] "RemoveContainer" containerID="f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7"
Jul 29 18:48:56.995075 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:48:56.995055    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7\": container with ID starting with f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7 not found: ID does not exist" containerID="f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7"
Jul 29 18:48:56.995122 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.995083    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7"} err="failed to get container status \"f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7\": rpc error: code = NotFound desc = could not find container \"f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7\": container with ID starting with f7b302180d003035ab50b82f643c9477ec8109ee9dacd1ec4d6d917ac1c0d9d7 not found: ID does not exist"
Jul 29 18:48:56.996428 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:56.996408    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"]
Jul 29 18:48:57.000380 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:57.000357    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8b8f04-openshift-default-5df5dfc468-mvsln"]
Jul 29 18:48:58.282534 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:48:58.282501    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" path="/var/lib/kubelet/pods/c6fe7b45-d52c-4081-9d57-51c2f2deebbe/volumes"
Jul 29 18:49:10.173127 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.173082    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"]
Jul 29 18:49:10.173579 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.173372    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw" podUID="554e07b2-c7dc-4deb-8227-e9442d4fdc9e" containerName="maas-api" containerID="cri-o://29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2" gracePeriod=30
Jul 29 18:49:10.425843 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.425765    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:49:10.530447 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.530417    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-openshift-service-ca\") pod \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") "
Jul 29 18:49:10.530602 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.530468    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-maas-api-tls\") pod \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") "
Jul 29 18:49:10.530602 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.530560    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ql2c\" (UniqueName: \"kubernetes.io/projected/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-kube-api-access-7ql2c\") pod \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\" (UID: \"554e07b2-c7dc-4deb-8227-e9442d4fdc9e\") "
Jul 29 18:49:10.530805 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.530776    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "554e07b2-c7dc-4deb-8227-e9442d4fdc9e" (UID: "554e07b2-c7dc-4deb-8227-e9442d4fdc9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:49:10.532654 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.532621    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-kube-api-access-7ql2c" (OuterVolumeSpecName: "kube-api-access-7ql2c") pod "554e07b2-c7dc-4deb-8227-e9442d4fdc9e" (UID: "554e07b2-c7dc-4deb-8227-e9442d4fdc9e"). InnerVolumeSpecName "kube-api-access-7ql2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:49:10.532745 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.532668    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "554e07b2-c7dc-4deb-8227-e9442d4fdc9e" (UID: "554e07b2-c7dc-4deb-8227-e9442d4fdc9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:49:10.631682 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.631646    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:49:10.631682 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.631678    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:49:10.631682 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:10.631687    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ql2c\" (UniqueName: \"kubernetes.io/projected/554e07b2-c7dc-4deb-8227-e9442d4fdc9e-kube-api-access-7ql2c\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:49:11.052137 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.052101    2560 generic.go:356] "Generic (PLEG): container finished" podID="554e07b2-c7dc-4deb-8227-e9442d4fdc9e" containerID="29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2" exitCode=0
Jul 29 18:49:11.052341 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.052152    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"
Jul 29 18:49:11.052341 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.052157    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw" event={"ID":"554e07b2-c7dc-4deb-8227-e9442d4fdc9e","Type":"ContainerDied","Data":"29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2"}
Jul 29 18:49:11.052341 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.052295    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw" event={"ID":"554e07b2-c7dc-4deb-8227-e9442d4fdc9e","Type":"ContainerDied","Data":"e80d2bb9e57f226fac54f5b3c2f90b8e835d9f0725a6b86f07ad98562125cf85"}
Jul 29 18:49:11.052341 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.052321    2560 scope.go:117] "RemoveContainer" containerID="29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2"
Jul 29 18:49:11.061535 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.061516    2560 scope.go:117] "RemoveContainer" containerID="29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2"
Jul 29 18:49:11.061782 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:49:11.061765    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2\": container with ID starting with 29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2 not found: ID does not exist" containerID="29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2"
Jul 29 18:49:11.061839 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.061790    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2"} err="failed to get container status \"29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2\": rpc error: code = NotFound desc = could not find container \"29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2\": container with ID starting with 29a694accd7084981a8cc121a024bd01b6c88cb1ec7d7ed7b1e0500721eb82e2 not found: ID does not exist"
Jul 29 18:49:11.072580 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.072557    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"]
Jul 29 18:49:11.078594 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:11.078561    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-479e5c-699848d957-d7mjw"]
Jul 29 18:49:12.282540 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:12.282500    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="554e07b2-c7dc-4deb-8227-e9442d4fdc9e" path="/var/lib/kubelet/pods/554e07b2-c7dc-4deb-8227-e9442d4fdc9e/volumes"
Jul 29 18:49:58.635976 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:58.635942    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"]
Jul 29 18:49:58.636513 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:49:58.636179    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy" containerID="cri-o://18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4" gracePeriod=30
Jul 29 18:50:03.891416 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.891390    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:50:03.940970 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.940882    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-envoy\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.940970 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.940940    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-certs\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.940970 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.940959    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-podinfo\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.941222 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941005    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istiod-ca-cert\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.941222 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941022    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-credential-socket\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.941222 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941078    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4h22\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-kube-api-access-m4h22\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.941222 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941122    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-socket\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.941222 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941171    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-data\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.941222 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941199    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-token\") pod \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\" (UID: \"1ed3be72-fbdf-4561-a5b7-7dab60583c67\") "
Jul 29 18:50:03.941563 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941222    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:03.941563 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941421    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:03.941563 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941463    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:50:03.941736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941606    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:03.941736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941628    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:03.941736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941641    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:03.941736 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.941644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:03.942412 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.942348    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:03.943392 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.943364    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:03.943520 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.943477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:50:03.943570 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.943537    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-kube-api-access-m4h22" (OuterVolumeSpecName: "kube-api-access-m4h22") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "kube-api-access-m4h22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:50:03.943613 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:03.943601    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ed3be72-fbdf-4561-a5b7-7dab60583c67" (UID: "1ed3be72-fbdf-4561-a5b7-7dab60583c67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:50:04.042850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.042812    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:04.042850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.042844    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:04.042850 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.042852    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:04.043091 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.042862    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:04.043091 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.042870    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed3be72-fbdf-4561-a5b7-7dab60583c67-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:04.043091 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.042883    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4h22\" (UniqueName: \"kubernetes.io/projected/1ed3be72-fbdf-4561-a5b7-7dab60583c67-kube-api-access-m4h22\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:04.344074 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.344042    2560 generic.go:356] "Generic (PLEG): container finished" podID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerID="18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4" exitCode=0
Jul 29 18:50:04.344283 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.344099    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"
Jul 29 18:50:04.344283 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.344129    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" event={"ID":"1ed3be72-fbdf-4561-a5b7-7dab60583c67","Type":"ContainerDied","Data":"18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4"}
Jul 29 18:50:04.344283 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.344180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl" event={"ID":"1ed3be72-fbdf-4561-a5b7-7dab60583c67","Type":"ContainerDied","Data":"5ee0ce0b91b99c9ac11e62c2cfc8e0128ca3ebf7244f19c6b47b23bc1475282f"}
Jul 29 18:50:04.344283 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.344202    2560 scope.go:117] "RemoveContainer" containerID="18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4"
Jul 29 18:50:04.353955 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.353936    2560 scope.go:117] "RemoveContainer" containerID="ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3"
Jul 29 18:50:04.360019 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.359997    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"]
Jul 29 18:50:04.363494 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.363473    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-479e5c-openshift-default-6dfc7c9c4d-c68hl"]
Jul 29 18:50:04.363660 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.363645    2560 scope.go:117] "RemoveContainer" containerID="18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4"
Jul 29 18:50:04.363924 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:50:04.363904    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4\": container with ID starting with 18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4 not found: ID does not exist" containerID="18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4"
Jul 29 18:50:04.363967 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.363933    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4"} err="failed to get container status \"18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4\": rpc error: code = NotFound desc = could not find container \"18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4\": container with ID starting with 18e82700e5acdace33f0c256738968288f90d23525078b60ebc6980b27499cd4 not found: ID does not exist"
Jul 29 18:50:04.363967 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.363964    2560 scope.go:117] "RemoveContainer" containerID="ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3"
Jul 29 18:50:04.364203 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:50:04.364186    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3\": container with ID starting with ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3 not found: ID does not exist" containerID="ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3"
Jul 29 18:50:04.364259 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:04.364210    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3"} err="failed to get container status \"ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3\": rpc error: code = NotFound desc = could not find container \"ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3\": container with ID starting with ca7f93afedc09e6b51e5ffd1a674a8184d50680384543391baca21aa1dea61b3 not found: ID does not exist"
Jul 29 18:50:06.288486 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:06.288444    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" path="/var/lib/kubelet/pods/1ed3be72-fbdf-4561-a5b7-7dab60583c67/volumes"
Jul 29 18:50:09.551162 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.551120    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"]
Jul 29 18:50:09.551636 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.551396    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5" podUID="14e419db-a9f8-496e-935d-530e4fbcef51" containerName="maas-api" containerID="cri-o://364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71" gracePeriod=30
Jul 29 18:50:09.807676 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.807616    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:50:09.901917 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.901881    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp2m9\" (UniqueName: \"kubernetes.io/projected/14e419db-a9f8-496e-935d-530e4fbcef51-kube-api-access-sp2m9\") pod \"14e419db-a9f8-496e-935d-530e4fbcef51\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") "
Jul 29 18:50:09.902116 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.901949    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e419db-a9f8-496e-935d-530e4fbcef51-maas-api-tls\") pod \"14e419db-a9f8-496e-935d-530e4fbcef51\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") "
Jul 29 18:50:09.902116 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.902060    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e419db-a9f8-496e-935d-530e4fbcef51-openshift-service-ca\") pod \"14e419db-a9f8-496e-935d-530e4fbcef51\" (UID: \"14e419db-a9f8-496e-935d-530e4fbcef51\") "
Jul 29 18:50:09.902493 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.902462    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14e419db-a9f8-496e-935d-530e4fbcef51-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14e419db-a9f8-496e-935d-530e4fbcef51" (UID: "14e419db-a9f8-496e-935d-530e4fbcef51"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:50:09.904123 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.904099    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e419db-a9f8-496e-935d-530e4fbcef51-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14e419db-a9f8-496e-935d-530e4fbcef51" (UID: "14e419db-a9f8-496e-935d-530e4fbcef51"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:50:09.904230 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:09.904126    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e419db-a9f8-496e-935d-530e4fbcef51-kube-api-access-sp2m9" (OuterVolumeSpecName: "kube-api-access-sp2m9") pod "14e419db-a9f8-496e-935d-530e4fbcef51" (UID: "14e419db-a9f8-496e-935d-530e4fbcef51"). InnerVolumeSpecName "kube-api-access-sp2m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:50:10.003200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.003160    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e419db-a9f8-496e-935d-530e4fbcef51-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:10.003200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.003192    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp2m9\" (UniqueName: \"kubernetes.io/projected/14e419db-a9f8-496e-935d-530e4fbcef51-kube-api-access-sp2m9\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:10.003200 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.003202    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e419db-a9f8-496e-935d-530e4fbcef51-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:10.380392 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.380356    2560 generic.go:356] "Generic (PLEG): container finished" podID="14e419db-a9f8-496e-935d-530e4fbcef51" containerID="364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71" exitCode=0
Jul 29 18:50:10.380552 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.380398    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"
Jul 29 18:50:10.380552 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.380442    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5" event={"ID":"14e419db-a9f8-496e-935d-530e4fbcef51","Type":"ContainerDied","Data":"364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71"}
Jul 29 18:50:10.380552 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.380499    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5" event={"ID":"14e419db-a9f8-496e-935d-530e4fbcef51","Type":"ContainerDied","Data":"5c738f5f6e958bf8676c95843a48ef5d2f2e5daf681d58c63935828c1b65ce27"}
Jul 29 18:50:10.380552 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.380527    2560 scope.go:117] "RemoveContainer" containerID="364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71"
Jul 29 18:50:10.392451 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.392430    2560 scope.go:117] "RemoveContainer" containerID="364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71"
Jul 29 18:50:10.392728 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:50:10.392707    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71\": container with ID starting with 364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71 not found: ID does not exist" containerID="364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71"
Jul 29 18:50:10.392800 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.392737    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71"} err="failed to get container status \"364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71\": rpc error: code = NotFound desc = could not find container \"364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71\": container with ID starting with 364b5b2e16350d52021b9d0b9a9f834b4dd5b318f0e47a93f4575c1174eebb71 not found: ID does not exist"
Jul 29 18:50:10.396959 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.396938    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"]
Jul 29 18:50:10.402729 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:10.402708    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b0fdf2-694c886d69-sxhd5"]
Jul 29 18:50:12.282725 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:12.282680    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14e419db-a9f8-496e-935d-530e4fbcef51" path="/var/lib/kubelet/pods/14e419db-a9f8-496e-935d-530e4fbcef51/volumes"
Jul 29 18:50:16.513085 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513047    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"]
Jul 29 18:50:16.513512 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513498    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14e419db-a9f8-496e-935d-530e4fbcef51" containerName="maas-api"
Jul 29 18:50:16.513558 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513517    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e419db-a9f8-496e-935d-530e4fbcef51" containerName="maas-api"
Jul 29 18:50:16.513558 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513536    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy"
Jul 29 18:50:16.513558 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513542    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy"
Jul 29 18:50:16.513558 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513554    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513562    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513569    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bffc5bc-47ca-4a71-a635-62abbe723b08" containerName="maas-api"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513575    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bffc5bc-47ca-4a71-a635-62abbe723b08" containerName="maas-api"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513585    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554e07b2-c7dc-4deb-8227-e9442d4fdc9e" containerName="maas-api"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513594    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="554e07b2-c7dc-4deb-8227-e9442d4fdc9e" containerName="maas-api"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513609    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513614    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513620    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513625    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513692    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bffc5bc-47ca-4a71-a635-62abbe723b08" containerName="maas-api"
Jul 29 18:50:16.513698 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513699    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy"
Jul 29 18:50:16.514054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513706    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy"
Jul 29 18:50:16.514054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513711    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="14e419db-a9f8-496e-935d-530e4fbcef51" containerName="maas-api"
Jul 29 18:50:16.514054 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.513721    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="554e07b2-c7dc-4deb-8227-e9442d4fdc9e" containerName="maas-api"
Jul 29 18:50:16.516865 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.516847    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"
Jul 29 18:50:16.518723 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.518700    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d5mcn\""
Jul 29 18:50:16.525634 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.525611    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"]
Jul 29 18:50:16.568140 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.568100    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15488065-502f-4c74-908a-be096749914d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm\" (UID: \"15488065-502f-4c74-908a-be096749914d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"
Jul 29 18:50:16.669760 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.669721    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15488065-502f-4c74-908a-be096749914d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm\" (UID: \"15488065-502f-4c74-908a-be096749914d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"
Jul 29 18:50:16.670364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.670346    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15488065-502f-4c74-908a-be096749914d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm\" (UID: \"15488065-502f-4c74-908a-be096749914d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"
Jul 29 18:50:16.829898 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.829810    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"
Jul 29 18:50:16.972301 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:16.972256    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"]
Jul 29 18:50:16.974734 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:50:16.974707    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15488065_502f_4c74_908a_be096749914d.slice/crio-5059cd5d9f4805f7230db7a3210cfcc59d100430b03c7832d1bd13303207bfff WatchSource:0}: Error finding container 5059cd5d9f4805f7230db7a3210cfcc59d100430b03c7832d1bd13303207bfff: Status 404 returned error can't find the container with id 5059cd5d9f4805f7230db7a3210cfcc59d100430b03c7832d1bd13303207bfff
Jul 29 18:50:17.419886 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:17.419844    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm" event={"ID":"15488065-502f-4c74-908a-be096749914d","Type":"ContainerStarted","Data":"405014d02e0073bc73611c27f2b6d42f2ae7b51b03c16cefc3c79bf1f18fe3c4"}
Jul 29 18:50:17.420074 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:17.419902    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm" event={"ID":"15488065-502f-4c74-908a-be096749914d","Type":"ContainerStarted","Data":"5059cd5d9f4805f7230db7a3210cfcc59d100430b03c7832d1bd13303207bfff"}
Jul 29 18:50:17.434156 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:17.434095    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm" podStartSLOduration=1.4340791689999999 podStartE2EDuration="1.434079169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:50:17.431971771 +0000 UTC m=+3407.673932808" watchObservedRunningTime="2026-07-29 18:50:17.434079169 +0000 UTC m=+3407.676040204"
Jul 29 18:50:18.426460 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:18.426420    2560 generic.go:356] "Generic (PLEG): container finished" podID="15488065-502f-4c74-908a-be096749914d" containerID="405014d02e0073bc73611c27f2b6d42f2ae7b51b03c16cefc3c79bf1f18fe3c4" exitCode=0
Jul 29 18:50:18.426853 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:18.426472    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm" event={"ID":"15488065-502f-4c74-908a-be096749914d","Type":"ContainerDied","Data":"405014d02e0073bc73611c27f2b6d42f2ae7b51b03c16cefc3c79bf1f18fe3c4"}
Jul 29 18:50:19.564945 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:19.564918    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"
Jul 29 18:50:19.698968 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:19.698874    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15488065-502f-4c74-908a-be096749914d-maas-api-service-ca\") pod \"15488065-502f-4c74-908a-be096749914d\" (UID: \"15488065-502f-4c74-908a-be096749914d\") "
Jul 29 18:50:19.699324 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:19.699303    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15488065-502f-4c74-908a-be096749914d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15488065-502f-4c74-908a-be096749914d" (UID: "15488065-502f-4c74-908a-be096749914d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:50:19.800235 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:19.800198    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15488065-502f-4c74-908a-be096749914d-maas-api-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:20.437606 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:20.437577    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"
Jul 29 18:50:20.437788 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:20.437617    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm" event={"ID":"15488065-502f-4c74-908a-be096749914d","Type":"ContainerDied","Data":"5059cd5d9f4805f7230db7a3210cfcc59d100430b03c7832d1bd13303207bfff"}
Jul 29 18:50:20.437788 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:20.437676    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5059cd5d9f4805f7230db7a3210cfcc59d100430b03c7832d1bd13303207bfff"
Jul 29 18:50:27.156768 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.156728    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"]
Jul 29 18:50:27.157234 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.156986    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2" podUID="c061dbd6-6581-4e28-b9f7-abf336927ff6" containerName="maas-api" containerID="cri-o://33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c" gracePeriod=30
Jul 29 18:50:27.360760 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.360724    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"]
Jul 29 18:50:27.362739 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.362710    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-297c4d-vx6dm"]
Jul 29 18:50:27.403529 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.403500    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:50:27.473743 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.473641    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061dbd6-6581-4e28-b9f7-abf336927ff6-openshift-service-ca\") pod \"c061dbd6-6581-4e28-b9f7-abf336927ff6\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") "
Jul 29 18:50:27.473743 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.473685    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w6gw\" (UniqueName: \"kubernetes.io/projected/c061dbd6-6581-4e28-b9f7-abf336927ff6-kube-api-access-6w6gw\") pod \"c061dbd6-6581-4e28-b9f7-abf336927ff6\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") "
Jul 29 18:50:27.473743 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.473737    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061dbd6-6581-4e28-b9f7-abf336927ff6-maas-api-tls\") pod \"c061dbd6-6581-4e28-b9f7-abf336927ff6\" (UID: \"c061dbd6-6581-4e28-b9f7-abf336927ff6\") "
Jul 29 18:50:27.474074 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.474050    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c061dbd6-6581-4e28-b9f7-abf336927ff6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c061dbd6-6581-4e28-b9f7-abf336927ff6" (UID: "c061dbd6-6581-4e28-b9f7-abf336927ff6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:50:27.476071 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.476041    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c061dbd6-6581-4e28-b9f7-abf336927ff6-kube-api-access-6w6gw" (OuterVolumeSpecName: "kube-api-access-6w6gw") pod "c061dbd6-6581-4e28-b9f7-abf336927ff6" (UID: "c061dbd6-6581-4e28-b9f7-abf336927ff6"). InnerVolumeSpecName "kube-api-access-6w6gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:50:27.476215 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.476091    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c061dbd6-6581-4e28-b9f7-abf336927ff6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c061dbd6-6581-4e28-b9f7-abf336927ff6" (UID: "c061dbd6-6581-4e28-b9f7-abf336927ff6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:50:27.476422 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.476400    2560 generic.go:356] "Generic (PLEG): container finished" podID="c061dbd6-6581-4e28-b9f7-abf336927ff6" containerID="33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c" exitCode=0
Jul 29 18:50:27.476476 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.476444    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"
Jul 29 18:50:27.476514 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.476474    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2" event={"ID":"c061dbd6-6581-4e28-b9f7-abf336927ff6","Type":"ContainerDied","Data":"33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c"}
Jul 29 18:50:27.476549 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.476527    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2" event={"ID":"c061dbd6-6581-4e28-b9f7-abf336927ff6","Type":"ContainerDied","Data":"c421ab22bee73424682c64c08c1de0c101af83c85c2b6f1fb68707398bffbe7c"}
Jul 29 18:50:27.476581 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.476548    2560 scope.go:117] "RemoveContainer" containerID="33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c"
Jul 29 18:50:27.495139 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.495119    2560 scope.go:117] "RemoveContainer" containerID="33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c"
Jul 29 18:50:27.495498 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:50:27.495478    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c\": container with ID starting with 33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c not found: ID does not exist" containerID="33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c"
Jul 29 18:50:27.495558 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.495507    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c"} err="failed to get container status \"33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c\": rpc error: code = NotFound desc = could not find container \"33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c\": container with ID starting with 33dbc3118247c73f0d2ef19634cb201f6a617d371709619e42b5cc99a2007c5c not found: ID does not exist"
Jul 29 18:50:27.504502 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.504480    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"]
Jul 29 18:50:27.510830 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.510806    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-297c4d-55d7bfccc9-zcdm2"]
Jul 29 18:50:27.575529 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.575498    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c061dbd6-6581-4e28-b9f7-abf336927ff6-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:27.575529 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.575525    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w6gw\" (UniqueName: \"kubernetes.io/projected/c061dbd6-6581-4e28-b9f7-abf336927ff6-kube-api-access-6w6gw\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:27.575723 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:27.575535    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c061dbd6-6581-4e28-b9f7-abf336927ff6-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:28.281846 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:28.281816    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15488065-502f-4c74-908a-be096749914d" path="/var/lib/kubelet/pods/15488065-502f-4c74-908a-be096749914d/volumes"
Jul 29 18:50:28.282214 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:28.282198    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c061dbd6-6581-4e28-b9f7-abf336927ff6" path="/var/lib/kubelet/pods/c061dbd6-6581-4e28-b9f7-abf336927ff6/volumes"
Jul 29 18:50:33.854411 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:33.854373    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"]
Jul 29 18:50:33.854938 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:33.854598    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy" containerID="cri-o://cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d" gracePeriod=30
Jul 29 18:50:39.137677 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.137649    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:50:39.186257 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186222    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-socket\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186291    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-credential-socket\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186318    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-podinfo\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186341    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6gb6\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-kube-api-access-f6gb6\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186365    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1806f65e-f061-4f46-9e45-071c6fd0d942-istiod-ca-cert\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186381    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-token\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186462 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186409    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-data\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186740 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186483    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-envoy\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186740 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186544    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-certs\") pod \"1806f65e-f061-4f46-9e45-071c6fd0d942\" (UID: \"1806f65e-f061-4f46-9e45-071c6fd0d942\") "
Jul 29 18:50:39.186740 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186562    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:39.186740 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186645    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:39.186948 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186802    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:39.186948 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186860    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1806f65e-f061-4f46-9e45-071c6fd0d942-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:50:39.186948 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186938    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-certs\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.187105 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186958    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-workload-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.187105 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186970    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-credential-socket\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.187105 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.186983    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1806f65e-f061-4f46-9e45-071c6fd0d942-istiod-ca-cert\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.187417 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.187391    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-data" (OuterVolumeSpecName: "istio-data") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:39.188714 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.188685    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:50:39.188814 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.188708    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-kube-api-access-f6gb6" (OuterVolumeSpecName: "kube-api-access-f6gb6") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "kube-api-access-f6gb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:50:39.188814 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.188762    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:50:39.189318 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.189299    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-token" (OuterVolumeSpecName: "istio-token") pod "1806f65e-f061-4f46-9e45-071c6fd0d942" (UID: "1806f65e-f061-4f46-9e45-071c6fd0d942"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:50:39.288079 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.288045    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-podinfo\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.288079 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.288071    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6gb6\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-kube-api-access-f6gb6\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.288079 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.288080    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-token\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.288351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.288094    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-data\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.288351 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.288106    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1806f65e-f061-4f46-9e45-071c6fd0d942-istio-envoy\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:39.541470 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.541433    2560 generic.go:356] "Generic (PLEG): container finished" podID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerID="cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d" exitCode=0
Jul 29 18:50:39.541652 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.541487    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"
Jul 29 18:50:39.541652 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.541515    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" event={"ID":"1806f65e-f061-4f46-9e45-071c6fd0d942","Type":"ContainerDied","Data":"cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d"}
Jul 29 18:50:39.541652 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.541569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp" event={"ID":"1806f65e-f061-4f46-9e45-071c6fd0d942","Type":"ContainerDied","Data":"bef6d4fe63f33e05a21a3bc419f2b00f830b3764cc3845c905631cb4caed70ff"}
Jul 29 18:50:39.541652 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.541591    2560 scope.go:117] "RemoveContainer" containerID="cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d"
Jul 29 18:50:39.551873 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.551854    2560 scope.go:117] "RemoveContainer" containerID="20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219"
Jul 29 18:50:39.560210 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.560184    2560 scope.go:117] "RemoveContainer" containerID="cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d"
Jul 29 18:50:39.560703 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:50:39.560673    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d\": container with ID starting with cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d not found: ID does not exist" containerID="cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d"
Jul 29 18:50:39.560825 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.560713    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d"} err="failed to get container status \"cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d\": rpc error: code = NotFound desc = could not find container \"cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d\": container with ID starting with cf30a2f30dc1b43ad83dae65b96e0cbbabef3f5c72d2b0d23972a86c5e35782d not found: ID does not exist"
Jul 29 18:50:39.560825 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.560739    2560 scope.go:117] "RemoveContainer" containerID="20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219"
Jul 29 18:50:39.561049 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:50:39.561017    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219\": container with ID starting with 20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219 not found: ID does not exist" containerID="20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219"
Jul 29 18:50:39.561049 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.561040    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219"} err="failed to get container status \"20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219\": rpc error: code = NotFound desc = could not find container \"20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219\": container with ID starting with 20a19a67629219146e5d4b0059cec540cf736dceb769c33cc93c4da83627e219 not found: ID does not exist"
Jul 29 18:50:39.563162 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.563140    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"]
Jul 29 18:50:39.565140 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:39.565116    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-297c4d-openshift-default-564c9f7496-nzzhp"]
Jul 29 18:50:40.282131 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:40.282100    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" path="/var/lib/kubelet/pods/1806f65e-f061-4f46-9e45-071c6fd0d942/volumes"
Jul 29 18:50:44.657210 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.657172    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"]
Jul 29 18:50:44.657712 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.657468    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7" podUID="f99c7104-600e-417a-a08d-2b282aed35f3" containerName="maas-api" containerID="cri-o://67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2" gracePeriod=30
Jul 29 18:50:44.910713 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.910650    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:50:44.940498 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.940466    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls\") pod \"f99c7104-600e-417a-a08d-2b282aed35f3\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") "
Jul 29 18:50:44.940670 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.940506    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f99c7104-600e-417a-a08d-2b282aed35f3-openshift-service-ca\") pod \"f99c7104-600e-417a-a08d-2b282aed35f3\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") "
Jul 29 18:50:44.940670 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.940536    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkhwz\" (UniqueName: \"kubernetes.io/projected/f99c7104-600e-417a-a08d-2b282aed35f3-kube-api-access-kkhwz\") pod \"f99c7104-600e-417a-a08d-2b282aed35f3\" (UID: \"f99c7104-600e-417a-a08d-2b282aed35f3\") "
Jul 29 18:50:44.940919 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.940895    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f99c7104-600e-417a-a08d-2b282aed35f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f99c7104-600e-417a-a08d-2b282aed35f3" (UID: "f99c7104-600e-417a-a08d-2b282aed35f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:50:44.942743 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.942714    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f99c7104-600e-417a-a08d-2b282aed35f3" (UID: "f99c7104-600e-417a-a08d-2b282aed35f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:50:44.942941 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:44.942916    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f99c7104-600e-417a-a08d-2b282aed35f3-kube-api-access-kkhwz" (OuterVolumeSpecName: "kube-api-access-kkhwz") pod "f99c7104-600e-417a-a08d-2b282aed35f3" (UID: "f99c7104-600e-417a-a08d-2b282aed35f3"). InnerVolumeSpecName "kube-api-access-kkhwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:50:45.041374 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.041342    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f99c7104-600e-417a-a08d-2b282aed35f3-maas-api-tls\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:45.041374 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.041371    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f99c7104-600e-417a-a08d-2b282aed35f3-openshift-service-ca\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:45.041592 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.041383    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkhwz\" (UniqueName: \"kubernetes.io/projected/f99c7104-600e-417a-a08d-2b282aed35f3-kube-api-access-kkhwz\") on node \"ip-10-0-130-240.ec2.internal\" DevicePath \"\""
Jul 29 18:50:45.573124 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.573086    2560 generic.go:356] "Generic (PLEG): container finished" podID="f99c7104-600e-417a-a08d-2b282aed35f3" containerID="67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2" exitCode=0
Jul 29 18:50:45.573315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.573130    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"
Jul 29 18:50:45.573315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.573177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7" event={"ID":"f99c7104-600e-417a-a08d-2b282aed35f3","Type":"ContainerDied","Data":"67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2"}
Jul 29 18:50:45.573315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.573233    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7" event={"ID":"f99c7104-600e-417a-a08d-2b282aed35f3","Type":"ContainerDied","Data":"6d3804aea697b4463b4760eeb5efbb524029781f0ec3ffe6bc016875800c403b"}
Jul 29 18:50:45.573315 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.573259    2560 scope.go:117] "RemoveContainer" containerID="67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2"
Jul 29 18:50:45.582135 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.582119    2560 scope.go:117] "RemoveContainer" containerID="67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2"
Jul 29 18:50:45.582438 ip-10-0-130-240 kubenswrapper[2560]: E0729 18:50:45.582416    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2\": container with ID starting with 67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2 not found: ID does not exist" containerID="67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2"
Jul 29 18:50:45.582487 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.582448    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2"} err="failed to get container status \"67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2\": rpc error: code = NotFound desc = could not find container \"67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2\": container with ID starting with 67cd5ac1c5a7fe2f4538b56391547a8594c3a1c76104c268d87be9c60dd2a3a2 not found: ID does not exist"
Jul 29 18:50:45.593685 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.593660    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"]
Jul 29 18:50:45.597290 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:45.597243    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f9520e-6c6bc69df8-7dww7"]
Jul 29 18:50:46.282297 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:46.282247    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f99c7104-600e-417a-a08d-2b282aed35f3" path="/var/lib/kubelet/pods/f99c7104-600e-417a-a08d-2b282aed35f3/volumes"
Jul 29 18:50:57.499872 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:57.499843    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7dc687fcc6-5msks_e19e8e60-6d2c-4345-9cbc-a5933c709313/manager/0.log"
Jul 29 18:50:57.844683 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:57.844602    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl_614b29cb-83fb-4fcd-824c-7d051959b106/copy-manifests/0.log"
Jul 29 18:50:57.852005 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:57.851977    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl_614b29cb-83fb-4fcd-824c-7d051959b106/manager/0.log"
Jul 29 18:50:58.423906 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:58.423876    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7b5699ddd4-qwp6v_7d38d4da-a4d2-4205-aadf-6b49268ef493/maas-api/0.log"
Jul 29 18:50:58.532639 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:58.532611    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755815-jkhkz_0485e84d-a778-4220-9cf2-7339e9780b15/cleanup/0.log"
Jul 29 18:50:58.641402 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:58.641370    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755830-9hbwj_94a9b25f-8529-46b8-8cef-d0a0644e3e70/cleanup/0.log"
Jul 29 18:50:58.749910 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:58.749888    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755845-sdjb8_98110a33-9998-4622-8be8-6079012239c5/cleanup/0.log"
Jul 29 18:50:58.854488 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:50:58.854453    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mtrrc_c05ad0b5-1385-4f7a-9572-c69d92abcd55/postgres/0.log"
Jul 29 18:51:00.447653 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:00.447622    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tjqdn_4a70595f-218a-49a2-96a3-5adee5fdfb57/kuadrant-console-plugin/0.log"
Jul 29 18:51:01.447206 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:01.447173    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-786b455d9f-c27kd_94bf186c-183c-4f7d-98f1-d8d787da0f82/kube-auth-proxy/0.log"
Jul 29 18:51:02.550789 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:02.550761    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mclp5_a0e7c809-dc66-4618-843a-ca196d8c1b1f/main/0.log"
Jul 29 18:51:02.657861 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:02.657829    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fdt2_47c341ee-d4be-4ec7-9d64-0c047bed8da7/main/0.log"
Jul 29 18:51:07.763942 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.763907    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-569bx/must-gather-d2g7q"]
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764292    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f99c7104-600e-417a-a08d-2b282aed35f3" containerName="maas-api"
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764305    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f99c7104-600e-417a-a08d-2b282aed35f3" containerName="maas-api"
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764314    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15488065-502f-4c74-908a-be096749914d" containerName="revoke-keys"
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764319    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15488065-502f-4c74-908a-be096749914d" containerName="revoke-keys"
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764325    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy"
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764331    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy"
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764354    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c061dbd6-6581-4e28-b9f7-abf336927ff6" containerName="maas-api"
Jul 29 18:51:07.764364 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764359    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c061dbd6-6581-4e28-b9f7-abf336927ff6" containerName="maas-api"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764420    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fe7b45-d52c-4081-9d57-51c2f2deebbe" containerName="istio-proxy"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764428    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764435    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ed3be72-fbdf-4561-a5b7-7dab60583c67" containerName="istio-proxy"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764441    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c061dbd6-6581-4e28-b9f7-abf336927ff6" containerName="maas-api"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764447    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764454    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="15488065-502f-4c74-908a-be096749914d" containerName="revoke-keys"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764460    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f99c7104-600e-417a-a08d-2b282aed35f3" containerName="maas-api"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764516    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy"
Jul 29 18:51:07.764621 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.764521    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1806f65e-f061-4f46-9e45-071c6fd0d942" containerName="istio-proxy"
Jul 29 18:51:07.769143 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.769126    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:07.771614 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.771592    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-569bx\"/\"openshift-service-ca.crt\""
Jul 29 18:51:07.772112 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.772092    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-569bx\"/\"default-dockercfg-ntx9j\""
Jul 29 18:51:07.772228 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.772133    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-569bx\"/\"kube-root-ca.crt\""
Jul 29 18:51:07.780930 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.780898    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-569bx/must-gather-d2g7q"]
Jul 29 18:51:07.858827 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.858783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/93887127-769f-4f36-9759-0c825bbe9514-must-gather-output\") pod \"must-gather-d2g7q\" (UID: \"93887127-769f-4f36-9759-0c825bbe9514\") " pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:07.858827 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.858828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mckfm\" (UniqueName: \"kubernetes.io/projected/93887127-769f-4f36-9759-0c825bbe9514-kube-api-access-mckfm\") pod \"must-gather-d2g7q\" (UID: \"93887127-769f-4f36-9759-0c825bbe9514\") " pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:07.959778 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.959738    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/93887127-769f-4f36-9759-0c825bbe9514-must-gather-output\") pod \"must-gather-d2g7q\" (UID: \"93887127-769f-4f36-9759-0c825bbe9514\") " pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:07.959778 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.959782    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mckfm\" (UniqueName: \"kubernetes.io/projected/93887127-769f-4f36-9759-0c825bbe9514-kube-api-access-mckfm\") pod \"must-gather-d2g7q\" (UID: \"93887127-769f-4f36-9759-0c825bbe9514\") " pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:07.960152 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.960129    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/93887127-769f-4f36-9759-0c825bbe9514-must-gather-output\") pod \"must-gather-d2g7q\" (UID: \"93887127-769f-4f36-9759-0c825bbe9514\") " pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:07.967075 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:07.967045    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mckfm\" (UniqueName: \"kubernetes.io/projected/93887127-769f-4f36-9759-0c825bbe9514-kube-api-access-mckfm\") pod \"must-gather-d2g7q\" (UID: \"93887127-769f-4f36-9759-0c825bbe9514\") " pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:08.080882 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:08.080788    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-569bx/must-gather-d2g7q"
Jul 29 18:51:08.215860 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:08.215834    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-569bx/must-gather-d2g7q"]
Jul 29 18:51:08.217193 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:51:08.217170    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93887127_769f_4f36_9759_0c825bbe9514.slice/crio-e44aea40af6b5bdf014d815693268384f72478ef20c23575906d8d614a12a88c WatchSource:0}: Error finding container e44aea40af6b5bdf014d815693268384f72478ef20c23575906d8d614a12a88c: Status 404 returned error can't find the container with id e44aea40af6b5bdf014d815693268384f72478ef20c23575906d8d614a12a88c
Jul 29 18:51:08.219046 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:08.219029    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:51:08.697220 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:08.697187    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-569bx/must-gather-d2g7q" event={"ID":"93887127-769f-4f36-9759-0c825bbe9514","Type":"ContainerStarted","Data":"e44aea40af6b5bdf014d815693268384f72478ef20c23575906d8d614a12a88c"}
Jul 29 18:51:09.706951 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:09.706899    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-569bx/must-gather-d2g7q" event={"ID":"93887127-769f-4f36-9759-0c825bbe9514","Type":"ContainerStarted","Data":"59c9755b41704b2f933cffee856141991e8bfecffd8952c5feb605a1d5b3975e"}
Jul 29 18:51:09.707576 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:09.706965    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-569bx/must-gather-d2g7q" event={"ID":"93887127-769f-4f36-9759-0c825bbe9514","Type":"ContainerStarted","Data":"4e35fd712dbf560840e36070abb79a1dfdb02e7f56f33c4e8c5cf305e3386cf9"}
Jul 29 18:51:09.723088 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:09.723022    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-569bx/must-gather-d2g7q" podStartSLOduration=1.9893891940000001 podStartE2EDuration="2.722999614s" totalImagesPullingTime="733.61042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:51:09.721011258 +0000 UTC m=+3459.962972300" watchObservedRunningTime="2026-07-29 18:51:09.722999614 +0000 UTC m=+3459.964960654"
Jul 29 18:51:10.499121 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:10.499094    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bphzv_df55af57-df49-40aa-9e88-a6f8e1a604cf/global-pull-secret-syncer/0.log"
Jul 29 18:51:10.664152 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:10.664097    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ksc6p_16fbc1a2-3b7b-44ac-8cba-d6f2b6753d12/konnectivity-agent/0.log"
Jul 29 18:51:10.738588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:10.738550    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-240.ec2.internal_a82b4de24054f43493689057761ee11c/haproxy/0.log"
Jul 29 18:51:16.462955 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:16.462925    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tjqdn_4a70595f-218a-49a2-96a3-5adee5fdfb57/kuadrant-console-plugin/0.log"
Jul 29 18:51:18.533102 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.533063    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ddf72b70-20e6-4650-8b39-20d2989ad867/alertmanager/0.log"
Jul 29 18:51:18.562027 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.561999    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ddf72b70-20e6-4650-8b39-20d2989ad867/config-reloader/0.log"
Jul 29 18:51:18.586291 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.586242    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ddf72b70-20e6-4650-8b39-20d2989ad867/kube-rbac-proxy-web/0.log"
Jul 29 18:51:18.611361 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.611320    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ddf72b70-20e6-4650-8b39-20d2989ad867/kube-rbac-proxy/0.log"
Jul 29 18:51:18.632857 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.632825    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ddf72b70-20e6-4650-8b39-20d2989ad867/kube-rbac-proxy-metric/0.log"
Jul 29 18:51:18.660208 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.660181    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ddf72b70-20e6-4650-8b39-20d2989ad867/prom-label-proxy/0.log"
Jul 29 18:51:18.687609 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.687562    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ddf72b70-20e6-4650-8b39-20d2989ad867/init-config-reloader/0.log"
Jul 29 18:51:18.760109 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.759980    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-zhg77_d803fc25-e93b-4d83-9257-05a40c2b3632/kube-state-metrics/0.log"
Jul 29 18:51:18.777297 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.777175    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-zhg77_d803fc25-e93b-4d83-9257-05a40c2b3632/kube-rbac-proxy-main/0.log"
Jul 29 18:51:18.799638 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.799559    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-zhg77_d803fc25-e93b-4d83-9257-05a40c2b3632/kube-rbac-proxy-self/0.log"
Jul 29 18:51:18.830899 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.830869    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-75f468f5b5-xgb9v_c8a9380c-bdaa-4797-88d0-f8e5c542e94e/metrics-server/0.log"
Jul 29 18:51:18.857477 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:18.857451    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-nq72d_95e69613-571d-4ed9-a3b9-6e3fbd8351be/monitoring-plugin/0.log"
Jul 29 18:51:19.016719 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.016690    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sbcc9_540c3ce9-8fb3-4ae7-8403-d3b704e63bb9/node-exporter/0.log"
Jul 29 18:51:19.048963 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.048937    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sbcc9_540c3ce9-8fb3-4ae7-8403-d3b704e63bb9/kube-rbac-proxy/0.log"
Jul 29 18:51:19.066069 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.065986    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sbcc9_540c3ce9-8fb3-4ae7-8403-d3b704e63bb9/init-textfile/0.log"
Jul 29 18:51:19.091588 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.091557    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-t6ns4_27eb9b6d-9114-4046-bb88-205f77a163fe/kube-rbac-proxy-main/0.log"
Jul 29 18:51:19.112360 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.112331    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-t6ns4_27eb9b6d-9114-4046-bb88-205f77a163fe/kube-rbac-proxy-self/0.log"
Jul 29 18:51:19.137768 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.137733    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-t6ns4_27eb9b6d-9114-4046-bb88-205f77a163fe/openshift-state-metrics/0.log"
Jul 29 18:51:19.168867 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.168839    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa/prometheus/0.log"
Jul 29 18:51:19.190429 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.190397    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa/config-reloader/0.log"
Jul 29 18:51:19.215572 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.215538    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa/thanos-sidecar/0.log"
Jul 29 18:51:19.239720 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.239682    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa/kube-rbac-proxy-web/0.log"
Jul 29 18:51:19.261184 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.261141    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa/kube-rbac-proxy/0.log"
Jul 29 18:51:19.284310 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.284258    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa/kube-rbac-proxy-thanos/0.log"
Jul 29 18:51:19.305929 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.305902    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e4eeb74e-b120-47b8-bb9e-fa0c7e5486aa/init-config-reloader/0.log"
Jul 29 18:51:19.337502 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.337414    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-cphv5_fc509132-6ae5-4fdc-a3a1-a057c4a17832/prometheus-operator/0.log"
Jul 29 18:51:19.354883 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.354851    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-cphv5_fc509132-6ae5-4fdc-a3a1-a057c4a17832/kube-rbac-proxy/0.log"
Jul 29 18:51:19.418021 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.417992    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f96947f78-5lzf2_c1913a48-ecb5-492e-a511-e3e2ec0342b3/telemeter-client/0.log"
Jul 29 18:51:19.443934 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.443902    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f96947f78-5lzf2_c1913a48-ecb5-492e-a511-e3e2ec0342b3/reload/0.log"
Jul 29 18:51:19.467061 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.467031    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f96947f78-5lzf2_c1913a48-ecb5-492e-a511-e3e2ec0342b3/kube-rbac-proxy/0.log"
Jul 29 18:51:19.498399 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.498369    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bd4688476-s6rrt_29557c5b-8b47-4cf6-b7e8-00a4e7e17cda/thanos-query/0.log"
Jul 29 18:51:19.519764 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.519710    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bd4688476-s6rrt_29557c5b-8b47-4cf6-b7e8-00a4e7e17cda/kube-rbac-proxy-web/0.log"
Jul 29 18:51:19.541672 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.541643    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bd4688476-s6rrt_29557c5b-8b47-4cf6-b7e8-00a4e7e17cda/kube-rbac-proxy/0.log"
Jul 29 18:51:19.563561 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.563537    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bd4688476-s6rrt_29557c5b-8b47-4cf6-b7e8-00a4e7e17cda/prom-label-proxy/0.log"
Jul 29 18:51:19.587917 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.587820    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bd4688476-s6rrt_29557c5b-8b47-4cf6-b7e8-00a4e7e17cda/kube-rbac-proxy-rules/0.log"
Jul 29 18:51:19.612083 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:19.612054    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bd4688476-s6rrt_29557c5b-8b47-4cf6-b7e8-00a4e7e17cda/kube-rbac-proxy-metrics/0.log"
Jul 29 18:51:20.673197 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.673155    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"]
Jul 29 18:51:20.679058 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.679021    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.686212 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.686186    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"]
Jul 29 18:51:20.716420 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.716382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwr5n\" (UniqueName: \"kubernetes.io/projected/87a82b9c-ae34-426d-b1fd-6163bffb42aa-kube-api-access-hwr5n\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.716420 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.716429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-sys\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.716669 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.716462    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-proc\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.716669 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.716482    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-podres\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.716669 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.716519    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-lib-modules\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.818942 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.818778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwr5n\" (UniqueName: \"kubernetes.io/projected/87a82b9c-ae34-426d-b1fd-6163bffb42aa-kube-api-access-hwr5n\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.818942 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.818832    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-sys\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.818942 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.818878    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-proc\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.818942 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.818910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-podres\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.819334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.818958    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-lib-modules\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.819334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.819136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-sys\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.819334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.819155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-lib-modules\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.819334 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.819254    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-podres\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.819548 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.819400    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87a82b9c-ae34-426d-b1fd-6163bffb42aa-proc\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.829301 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.829256    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwr5n\" (UniqueName: \"kubernetes.io/projected/87a82b9c-ae34-426d-b1fd-6163bffb42aa-kube-api-access-hwr5n\") pod \"perf-node-gather-daemonset-z22wb\" (UID: \"87a82b9c-ae34-426d-b1fd-6163bffb42aa\") " pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:20.994636 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:20.994606    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:21.189882 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:21.189823    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"]
Jul 29 18:51:21.197828 ip-10-0-130-240 kubenswrapper[2560]: W0729 18:51:21.195042    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod87a82b9c_ae34_426d_b1fd_6163bffb42aa.slice/crio-6e86eed732843b83756f1bd84881917e9fa374f2329822c8c354b68f83edaf95 WatchSource:0}: Error finding container 6e86eed732843b83756f1bd84881917e9fa374f2329822c8c354b68f83edaf95: Status 404 returned error can't find the container with id 6e86eed732843b83756f1bd84881917e9fa374f2329822c8c354b68f83edaf95
Jul 29 18:51:21.810752 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:21.810708    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb" event={"ID":"87a82b9c-ae34-426d-b1fd-6163bffb42aa","Type":"ContainerStarted","Data":"3b5a46cd742b4259ff2a3bcc7f26abecb82dc3bfdd5c6c82aa62b4f29c3b9f18"}
Jul 29 18:51:21.811244 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:21.810789    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb" event={"ID":"87a82b9c-ae34-426d-b1fd-6163bffb42aa","Type":"ContainerStarted","Data":"6e86eed732843b83756f1bd84881917e9fa374f2329822c8c354b68f83edaf95"}
Jul 29 18:51:21.811244 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:21.810821    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:23.637892 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:23.637855    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b5xhl_859ad4b1-4e5f-4677-990b-aa063d8d75ae/dns/0.log"
Jul 29 18:51:23.658932 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:23.658897    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b5xhl_859ad4b1-4e5f-4677-990b-aa063d8d75ae/kube-rbac-proxy/0.log"
Jul 29 18:51:23.769309 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:23.769260    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tlm66_ecbaf110-7713-4aed-b3ad-78f77508ceb5/dns-node-resolver/0.log"
Jul 29 18:51:24.361890 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:24.361860    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g9f6g_25a9391d-00df-4c40-80cc-137625065288/node-ca/0.log"
Jul 29 18:51:25.462106 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:25.462075    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-786b455d9f-c27kd_94bf186c-183c-4f7d-98f1-d8d787da0f82/kube-auth-proxy/0.log"
Jul 29 18:51:26.422014 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:26.421981    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2847d_15ec954f-65e8-4d47-b88a-4fd8d418ae26/serve-healthcheck-canary/0.log"
Jul 29 18:51:26.966428 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:26.966400    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmfxl_1bcea560-650a-41e7-94cf-69189935446f/kube-rbac-proxy/0.log"
Jul 29 18:51:27.006160 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:27.006130    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmfxl_1bcea560-650a-41e7-94cf-69189935446f/exporter/0.log"
Jul 29 18:51:27.042206 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:27.042178    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmfxl_1bcea560-650a-41e7-94cf-69189935446f/extractor/0.log"
Jul 29 18:51:27.840616 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:27.840341    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb"
Jul 29 18:51:27.857290 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:27.857228    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-569bx/perf-node-gather-daemonset-z22wb" podStartSLOduration=7.857212304 podStartE2EDuration="7.857212304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:51:21.832896312 +0000 UTC m=+3472.074857351" watchObservedRunningTime="2026-07-29 18:51:27.857212304 +0000 UTC m=+3478.099173343"
Jul 29 18:51:29.847816 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:29.847783    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7dc687fcc6-5msks_e19e8e60-6d2c-4345-9cbc-a5933c709313/manager/0.log"
Jul 29 18:51:29.974591 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:29.974555    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl_614b29cb-83fb-4fcd-824c-7d051959b106/manager/0.log"
Jul 29 18:51:29.995575 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:29.995532    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bc8dd9bf6-btwnl_614b29cb-83fb-4fcd-824c-7d051959b106/copy-manifests/0.log"
Jul 29 18:51:31.486162 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:31.486080    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8569878f48-bb6zg_90ce4563-ded9-4246-97e4-eb12b8e91850/manager/0.log"
Jul 29 18:51:31.654145 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:31.654115    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-786b455d9f-c27kd_94bf186c-183c-4f7d-98f1-d8d787da0f82/kube-auth-proxy/0.log"
Jul 29 18:51:33.410574 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:33.410542    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-786b455d9f-c27kd_94bf186c-183c-4f7d-98f1-d8d787da0f82/kube-auth-proxy/0.log"
Jul 29 18:51:38.178372 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:38.178335    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkbq8_2dc05f89-a229-4b2b-9d29-4d3e2521b691/kube-multus-additional-cni-plugins/0.log"
Jul 29 18:51:38.199447 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:38.199415    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkbq8_2dc05f89-a229-4b2b-9d29-4d3e2521b691/egress-router-binary-copy/0.log"
Jul 29 18:51:38.219989 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:38.219963    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkbq8_2dc05f89-a229-4b2b-9d29-4d3e2521b691/cni-plugins/0.log"
Jul 29 18:51:38.241115 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:38.241084    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkbq8_2dc05f89-a229-4b2b-9d29-4d3e2521b691/bond-cni-plugin/0.log"
Jul 29 18:51:38.261312 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:38.261278    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkbq8_2dc05f89-a229-4b2b-9d29-4d3e2521b691/routeoverride-cni/0.log"
Jul 29 18:51:38.282526 ip-10-0-130-240 kubenswrapper[2560]: I0729 18:51:38.282497    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tkbq8_2dc05f89-a229-4b2b-9d29-4d3e2521b691/whereabouts-cni-bincopy/0.log"