Jul 15 12:07:48.778398 ip-10-0-135-186 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.227401    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231498    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231512    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231517    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231524    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231529    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231533    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:07:49.274153 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231536    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231538    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231541    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231544    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231546    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231549    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231551    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231554    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231564    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231567    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231570    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231572    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231575    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231578    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231580    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231583    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231586    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231589    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231592    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231594    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:07:49.276714 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231597    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231600    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231603    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231606    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231610    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231614    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231617    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231620    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231622    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231632    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231634    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231637    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231639    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231642    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231644    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231647    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231650    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231653    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231655    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231658    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:07:49.279273 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231667    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231670    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231673    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231676    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231679    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231681    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231684    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231687    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231689    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231692    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231694    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231697    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231699    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231701    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231704    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231706    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231709    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231712    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231714    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231716    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:07:49.287306 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231719    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231721    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231724    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231727    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231730    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231732    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231737    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231740    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231743    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231745    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231748    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231750    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231753    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231761    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231764    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231767    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231769    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231772    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231775    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:07:49.288757 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231778    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231780    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231783    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231786    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231788    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231791    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231793    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231796    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231799    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231801    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231805    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231809    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231812    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.231814    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232553    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232561    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232564    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232567    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232570    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:07:49.291975 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232573    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232575    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232579    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232581    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232584    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232586    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232589    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232592    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232595    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232597    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232600    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232603    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232605    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232608    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232610    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232613    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232618    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232621    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232624    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232627    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:07:49.294187 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232630    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232633    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232635    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232638    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232641    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232644    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232648    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232651    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232653    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232656    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232658    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232661    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232663    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232666    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232668    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232671    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232674    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232676    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232679    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:07:49.313777 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232682    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:07:49.306487 ip-10-0-135-186 systemd[1]: Started Kubernetes Kubelet.
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232685    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232688    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232690    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232693    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232695    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232698    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232701    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232704    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232707    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232709    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232712    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232714    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232718    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232721    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232723    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232726    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232728    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232731    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232734    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:07:49.314900 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232736    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232739    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232743    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232746    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232749    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232752    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232755    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232759    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232763    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232766    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232769    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232772    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232774    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232777    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232784    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232793    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232796    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232799    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232802    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:07:49.317736 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232804    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232807    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232810    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232812    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232815    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232817    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232820    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232823    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232825    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232828    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232830    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232833    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232836    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232838    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232841    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.232844    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233313    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233319    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233322    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233324    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:07:49.318802 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233327    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233330    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233332    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233336    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233340    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233344    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233346    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233350    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233358    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233361    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233363    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233366    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233368    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233371    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233373    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233376    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233378    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233381    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233384    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:07:49.321196 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233386    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233389    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233391    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233394    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233396    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233401    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233404    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233406    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233409    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233411    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233414    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233420    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233424    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233427    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233429    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233432    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233435    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233438    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233440    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:07:49.322216 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233443    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233445    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233448    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233455    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233458    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233461    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233463    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233466    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233468    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233471    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233473    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233476    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233478    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233481    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233484    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233487    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233489    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233493    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233497    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:07:49.324958 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233500    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233503    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233506    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233510    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233512    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233516    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233519    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233522    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233524    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233527    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233529    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233532    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233534    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233537    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233540    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233542    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233545    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233553    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233555    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233558    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:07:49.325938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233561    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233563    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233565    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233568    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233571    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233573    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233575    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233578    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233580    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233583    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233585    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233589    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233591    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233594    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233596    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233599    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233601    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.233605    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233694    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233705    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233722    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 12:07:49.329604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233726    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233730    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233734    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233742    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233746    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233749    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233752    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233756    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233759    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233768    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233771    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233774    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233777    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233780    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233783    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233789    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233792    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233795    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233798    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233801    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233805    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233808    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233813    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 12:07:49.330953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233816    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233819    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233822    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233825    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233828    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233833    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233837    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233839    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233842    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233845    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233848    2582 flags.go:64] FLAG: --enable-server="true"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233851    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233858    2582 flags.go:64] FLAG: --event-burst="100"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233861    2582 flags.go:64] FLAG: --event-qps="50"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233864    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233867    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233870    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233873    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233876    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233884    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233887    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233890    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233893    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233896    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233899    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 12:07:49.334300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233902    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233905    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233907    2582 flags.go:64] FLAG: --feature-gates=""
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233911    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233919    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233923    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233927    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233930    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233933    2582 flags.go:64] FLAG: --help="false"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233936    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-135-186.ec2.internal"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233939    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233941    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233946    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233950    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233953    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233956    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233959    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233962    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233964    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233967    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233970    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233973    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233976    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233978    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 12:07:49.335781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233981    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.233984    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234000    2582 flags.go:64] FLAG: --lock-file=""
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234009    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234013    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234016    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234021    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234024    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234026    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234029    2582 flags.go:64] FLAG: --logging-format="text"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234032    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234035    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234038    2582 flags.go:64] FLAG: --manifest-url=""
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234041    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234046    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234050    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234054    2582 flags.go:64] FLAG: --max-pods="110"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234056    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234059    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234062    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234066    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234069    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234072    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234075    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234082    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 12:07:49.340356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234085    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234088    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234107    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234110    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234115    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234118    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234121    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234123    2582 flags.go:64] FLAG: --port="10250"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234126    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234130    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04aa180ed92920ef8"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234133    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234143    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234146    2582 flags.go:64] FLAG: --register-node="true"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234149    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234152    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234155    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234158    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234161    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234164    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234167    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234170    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234173    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234177    2582 flags.go:64] FLAG: --runonce="false"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234180    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234183    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 12:07:49.341821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234186    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234189    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234193    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234196    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234199    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234202    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234204    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234207    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234209    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234212    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234215    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234218    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234221    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234226    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234228    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234231    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234237    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234240    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234243    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234251    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234254    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234257    2582 flags.go:64] FLAG: --v="2"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234260    2582 flags.go:64] FLAG: --version="false"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234264    2582 flags.go:64] FLAG: --vmodule=""
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234267    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 12:07:49.444765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.234271    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234435    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234439    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234442    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234446    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234449    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234452    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234455    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234457    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234462    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234466    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234469    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234472    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234474    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234477    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234479    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234482    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234484    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234487    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:07:49.446255 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234489    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234492    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234496    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234499    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234502    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234505    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234508    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234511    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234520    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234523    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234526    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234528    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234530    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234533    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234536    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234538    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234541    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234545    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234548    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:07:49.449882 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234550    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234559    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234562    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234566    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234569    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234571    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234574    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234576    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234579    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234581    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234584    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234587    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234589    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234592    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234594    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234597    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234599    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234603    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234606    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:07:49.450883 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234609    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234612    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234615    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234623    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234625    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234628    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234630    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234633    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234636    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234638    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234641    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234644    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234647    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234650    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234652    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234655    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234660    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234663    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234665    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234668    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:07:49.454071 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234670    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234673    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234675    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234678    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234680    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234683    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234685    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234688    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234690    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234693    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234695    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234698    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234701    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234703    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234706    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234708    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234718    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234722    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234724    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234726    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:07:49.455053 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234729    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234731    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.234734    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.235455    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.243362    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.243379    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.243398    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.243405    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243462    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243468    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243471    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243474    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243477    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243480    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243483    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:07:49.459594 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243486    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243489    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243491    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243494    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243496    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243499    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243502    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243505    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243507    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243510    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243513    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243515    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243518    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243521    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243525    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243529    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243532    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243535    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243538    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243541    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:07:49.460578 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243544    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243548    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243551    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243554    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243556    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243559    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243561    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243564    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243567    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243569    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243572    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243575    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243577    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243580    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243582    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243585    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243587    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243590    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243592    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243595    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:07:49.465161 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243597    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243600    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243602    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243605    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243607    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243610    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243613    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243616    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243620    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243623    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243626    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243629    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243632    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243635    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243638    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243641    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243644    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243646    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243649    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243652    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:07:49.466188 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243654    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243657    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243659    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243662    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243664    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243667    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243669    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243672    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243674    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243677    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243679    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243682    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243685    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243687    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243691    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243695    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243698    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243700    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243703    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:07:49.469786 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243705    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243708    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243710    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243713    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243715    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243718    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243721    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243724    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243727    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243729    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243732    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243735    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243737    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.243742    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243851    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:07:49.470778 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243856    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243859    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243862    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243864    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243867    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243870    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243872    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243875    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243877    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243880    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243883    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243886    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243889    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243891    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243894    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243896    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243899    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243901    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243904    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243906    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:07:49.473936 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243912    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243914    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243917    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243920    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243922    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243925    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243927    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243930    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243933    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243935    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243939    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243943    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243946    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243950    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243952    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243955    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243958    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243960    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243963    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:07:49.474985 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243966    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243969    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243971    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243974    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243977    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243982    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243984    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243987    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243990    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243992    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243995    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.243998    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244000    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244004    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244006    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244009    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244011    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244014    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244017    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244019    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:07:49.479637 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244021    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244024    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244028    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244032    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244034    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244037    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244039    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244042    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244044    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244047    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244049    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244053    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244056    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244059    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244062    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244064    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244067    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244072    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244075    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:07:49.480638 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244077    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244080    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244082    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244085    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244087    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244108    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244112    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244115    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244117    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244120    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244123    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244125    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244128    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244130    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244133    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244136    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244138    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244141    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244143    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:07:49.584405 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:49.244146    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.244151    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.244266    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.247813    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.249210    2582 server.go:1019] "Starting client certificate rotation"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.249312    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.249349    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.251837    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.253122    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.265548    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.268026    2582 log.go:25] "Validated CRI v1 image API"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.270712    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 12:07:49.585362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.276235    2582 fs.go:135] Filesystem UUIDs: map[54b136ff-f949-4fb0-bc32-3781dd5c518a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ce6f526f-247e-453b-bc39-675e603a0712:/dev/nvme0n1p3]
Jul 15 12:07:49.586335 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.276250    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 12:07:49.590717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.283539    2582 manager.go:216] Machine: {Timestamp:2026-07-15 12:07:49.28003897 +0000 UTC m=+0.379988304 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101729 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec200bece79e6ad8688acc721906847d SystemUUID:ec200bec-e79e-6ad8-688a-cc721906847d BootID:04093476-dfbc-4e5e-b0f9-269907c2990a Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:70:22:88:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:70:22:88:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:db:1c:b5:cd:2f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 12:07:49.590717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.283648    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 12:07:49.590717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.283732    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 12:07:49.590717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.286645    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 12:07:49.590717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.287064    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 12:07:49.590717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.287107    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 12:07:49.590717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.290015    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.290027    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.290035    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.291640    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.292436    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295187    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295337    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295365    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295374    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295389    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295399    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295411    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295819    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.295834    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.298689    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.300386    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.300421    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301325    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301341    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301349    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301358    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301369    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301376    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301386    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301394    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301402    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301410    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301428    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.301435    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.302288    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.302299    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.305835    2582 server.go:1268] "Started kubelet"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.306178    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.307738    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.308437    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.308469    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.309654    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.313135    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.313212    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.314369    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.314755    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.316866    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:49.591218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.317363    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.317380    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.317487    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.317772    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318352    2582 factory.go:55] Registering systemd factory
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318370    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318371    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318383    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318603    2582 factory.go:153] Registering CRI-O factory
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318612    2582 factory.go:223] Registration of the crio container factory successfully
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318669    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318689    2582 factory.go:103] Registering Raw factory
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.318698    2582 manager.go:1196] Started watching for new ooms in manager
Jul 15 12:07:49.593256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.319202    2582 manager.go:349] Starting recovery of all containers
Jul 15 12:07:49.708997 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.699820    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:49.708997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.702839    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jslnt"
Jul 15 12:07:49.708997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.702864    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jslnt"
Jul 15 12:07:49.708997 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.706885    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 12:07:49.708997 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.706898    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 12:07:49.708997 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.707082    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 12:07:49.823807 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.706955    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-186.ec2.internal.18c275342a9abd77  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-186.ec2.internal,UID:ip-10-0-135-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-186.ec2.internal,},FirstTimestamp:2026-07-15 12:07:49.305671031 +0000 UTC m=+0.405620362,LastTimestamp:2026-07-15 12:07:49.305671031 +0000 UTC m=+0.405620362,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-186.ec2.internal,}"
Jul 15 12:07:49.823807 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.800441    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:49.869611 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.869531    2582 manager.go:354] Recovery completed
Jul 15 12:07:49.874147 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.874129    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:07:49.877109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.877090    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:07:49.877198 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.877139    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:07:49.877198 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.877156    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:07:49.877703 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.877689    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 12:07:49.877748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.877714    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 12:07:49.877748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.877734    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 12:07:49.877814 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.877701    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:07:49.878709 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.878692    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:07:49.878762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.878718    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:07:49.878762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.878729    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:07:49.878993 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.878980    2582 policy_none.go:49] "None policy: Start"
Jul 15 12:07:49.879026 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.878999    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 12:07:49.879026 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.879018    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 12:07:49.879947 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.879842    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-186.ec2.internal.18c275344caa6521  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-186.ec2.internal,UID:ip-10-0-135-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-186.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-186.ec2.internal,},FirstTimestamp:2026-07-15 12:07:49.877122337 +0000 UTC m=+0.977071674,LastTimestamp:2026-07-15 12:07:49.877122337 +0000 UTC m=+0.977071674,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-186.ec2.internal,}"
Jul 15 12:07:49.880252 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.880239    2582 policy_none.go:47] "Start"
Jul 15 12:07:49.901389 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.901272    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.914312    2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-186.ec2.internal\" not found" node="ip-10-0-135-186.ec2.internal"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.917793    2582 manager.go:337] "Starting Device Plugin manager"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.917834    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.917844    2582 server.go:87] "Starting device plugin registration server"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.918172    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.918185    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.918322    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.918413    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:49.918423    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.919340    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 12:07:49.923893 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:49.919383    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.018977 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.018947    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:07:50.019896 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.019881    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:07:50.019970 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.019926    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:07:50.019970 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.019939    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:07:50.019970 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.019970    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.030943 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.030928    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.031006 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.030953    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-186.ec2.internal\": node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.054699 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.054664    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 12:07:50.055988 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.055969    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 12:07:50.056077 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.056014    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 12:07:50.056077 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.056052    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 12:07:50.056154 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.056092    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 12:07:50.061808 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.061789    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:07:50.070904 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.070887    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.157712 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.157624    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"]
Jul 15 12:07:50.157805 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.157722    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:07:50.158889 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.158868    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:07:50.158951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.158911    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:07:50.158951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.158926    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:07:50.160364 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.160350    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:07:50.160540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.160525    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.160578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.160569    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:07:50.161151 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.161128    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:07:50.161237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.161157    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:07:50.161237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.161160    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:07:50.161237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.161181    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:07:50.161237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.161190    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:07:50.161237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.161191    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:07:50.162431 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.162418    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.162484 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.162445    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:07:50.163185 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.163170    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:07:50.163259 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.163197    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:07:50.163259 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.163208    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:07:50.171282 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.171262    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.202950 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.202924    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-186.ec2.internal\" not found" node="ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.207500 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.207485    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-186.ec2.internal\" not found" node="ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.240118 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.240093    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:07:50.248652 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.248627    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 12:07:50.248783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.248743    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 12:07:50.248837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.248745    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 12:07:50.248879 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.248784    2582 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a8d39518f970e4511bf51f6b74e0c5e4-3c6f82d6115ea7e2.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-135-186.ec2.internal.18c275344caa6521\": read tcp 10.0.135.186:57750->32.193.47.180:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-135-186.ec2.internal.18c275344caa6521  default   5682 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-186.ec2.internal,UID:ip-10-0-135-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-186.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-186.ec2.internal,},FirstTimestamp:2026-07-15 12:07:49 +0000 UTC,LastTimestamp:2026-07-15 12:07:50.163183801 +0000 UTC m=+1.263133134,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-186.ec2.internal,}"
Jul 15 12:07:50.272656 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.272632    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.309151 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.309119    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b45ab77f7adac80ae4f46c80d27c83b4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-186.ec2.internal\" (UID: \"b45ab77f7adac80ae4f46c80d27c83b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.309555 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.309155    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.309555 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.309173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.374825 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.374798    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.410229 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.410167    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b45ab77f7adac80ae4f46c80d27c83b4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-186.ec2.internal\" (UID: \"b45ab77f7adac80ae4f46c80d27c83b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.410229 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.410207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.410229 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.410226    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.410403 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.410285    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b45ab77f7adac80ae4f46c80d27c83b4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-186.ec2.internal\" (UID: \"b45ab77f7adac80ae4f46c80d27c83b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.410403 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.410309    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.410403 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.410344    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.479695 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.479662    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.505481 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.505441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.510911 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.510891    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:50.583238 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.583202    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.684482 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.684413    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.785592 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.785565    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.867501 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.867457    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 12:02:49 +0000 UTC" deadline="2028-01-01 18:53:03.69238346 +0000 UTC"
Jul 15 12:07:50.867501 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:50.867492    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12846h45m12.824895376s"
Jul 15 12:07:50.887069 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.887045    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:50.988054 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:50.987981    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:51.088656 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.088623    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:51.097415 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.097386    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa7a67259a3462e5ba6775b001f3248.slice/crio-693175539b9e27a125044eb6139a9ad629fb61c0e87094ae7d5d082f6f867fa8 WatchSource:0}: Error finding container 693175539b9e27a125044eb6139a9ad629fb61c0e87094ae7d5d082f6f867fa8: Status 404 returned error can't find the container with id 693175539b9e27a125044eb6139a9ad629fb61c0e87094ae7d5d082f6f867fa8
Jul 15 12:07:51.101302 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.101284    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:07:51.142829 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.142803    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:07:51.178165 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.178147    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:07:51.188950 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.188923    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found"
Jul 15 12:07:51.197988 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.197970    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:07:51.218071 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.218051    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:51.229768 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.229745    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 12:07:51.230776 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.230763    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal"
Jul 15 12:07:51.247966 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.247914    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 12:07:51.295964 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.295944    2582 apiserver.go:52] "Watching apiserver"
Jul 15 12:07:51.301877 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.301860    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 12:07:51.303242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.303219    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp","openshift-cluster-node-tuning-operator/tuned-8tfkf","openshift-multus/multus-additional-cni-plugins-qh85v","openshift-multus/multus-c2d9c","openshift-network-diagnostics/network-check-target-4qbcd","openshift-network-operator/iptables-alerter-rhl8m","openshift-ovn-kubernetes/ovnkube-node-9sx9r","kube-system/global-pull-secret-syncer-h4nz6","kube-system/konnectivity-agent-49slm","openshift-dns/node-resolver-7bkzq","openshift-image-registry/node-ca-sp6l2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal","openshift-multus/network-metrics-daemon-9jcw5"]
Jul 15 12:07:51.305427 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.305410    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.306660 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.306644    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.308005 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.307979    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.308109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.308082    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 12:07:51.308109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.308088    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 12:07:51.308244 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.308161    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:07:51.308356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.308288    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-754sv\""
Jul 15 12:07:51.308638 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.308621    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kz78l\""
Jul 15 12:07:51.308895 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.308880    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 12:07:51.308969 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.308953    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 12:07:51.310812 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.310794    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.311411 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.311063    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 12:07:51.311411 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.311119    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:07:51.311411 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.311123    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g7hgb\""
Jul 15 12:07:51.311981 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.311966    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.313118 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.312842    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 12:07:51.313118 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.312936    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 12:07:51.313118 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.312957    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 12:07:51.314051 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.313856    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 12:07:51.314051 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.313900    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:51.314235 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.314077    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:07:51.314618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.314368    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 12:07:51.314618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.314429    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xrjfp\""
Jul 15 12:07:51.314618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.314443    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7kvpd\""
Jul 15 12:07:51.314618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.314460    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 12:07:51.317066 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.317042    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.317162 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.317143    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:07:51.317273 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.317257    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 12:07:51.318429 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.318404    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:51.318540 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.318464    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:07:51.319654 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.319638    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.320764 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320746    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-conf-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.320869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320770    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-modprobe-d\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.320869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320787    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysconfig\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.320869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320802    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-kubernetes\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.320869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-systemd\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.320869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320862    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-multus-certs\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dac464eb-a63f-470b-883d-93d07e830acb-dbus\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320911    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wq4k\" (UniqueName: \"kubernetes.io/projected/cae9499a-36e9-4a31-aba3-c268211cc3a9-kube-api-access-9wq4k\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320928    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320950    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysctl-conf\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320965    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-run\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-sys\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.320993    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-host\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321015    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-daemon-config\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321057    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-system-cni-dir\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321081    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-var-lib-kubelet\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321095    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/462ed658-e936-483e-8353-cde6d14feec3-tmp\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321118    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-system-cni-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-cni-multus\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321180    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-cnibin\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321314    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321341    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-cni-binary-copy\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321435    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-netns\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-cni-bin\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321511    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321552    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321600    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7df60601-9a45-400a-8937-5f310dc9e70e-host-slash\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5749n\" (UniqueName: \"kubernetes.io/projected/462ed658-e936-483e-8353-cde6d14feec3-kube-api-access-5749n\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321650    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-cni-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-cnibin\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.321752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-socket-dir-parent\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-k8s-cni-cncf-io\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj5vz\" (UniqueName: \"kubernetes.io/projected/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-kube-api-access-mj5vz\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7df60601-9a45-400a-8937-5f310dc9e70e-iptables-alerter-script\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-lib-modules\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321837    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321863    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67x9\" (UniqueName: \"kubernetes.io/projected/7df60601-9a45-400a-8937-5f310dc9e70e-kube-api-access-s67x9\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321890    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/462ed658-e936-483e-8353-cde6d14feec3-etc-tuned\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321915    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-kubelet\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321971    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.321975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-etc-kubernetes\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dac464eb-a63f-470b-883d-93d07e830acb-kubelet-config\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322023    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.322483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322045    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-os-release\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg5rc\" (UniqueName: \"kubernetes.io/projected/8ee8bd80-e264-413e-84d3-1a40c8c412ca-kube-api-access-lg5rc\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322097    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/728cb636-c45f-42d8-a03b-b5f54a4fd18d-hosts-file\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322120    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728cb636-c45f-42d8-a03b-b5f54a4fd18d-tmp-dir\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l6b2\" (UniqueName: \"kubernetes.io/projected/728cb636-c45f-42d8-a03b-b5f54a4fd18d-kube-api-access-6l6b2\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322181    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d7xbs\""
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysctl-d\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-os-release\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322261    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-hostroot\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322290    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 12:07:51.322957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.322703    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.323480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.323346    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9nqnj\""
Jul 15 12:07:51.323480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.323357    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 12:07:51.323480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.323380    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 12:07:51.324081 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.324063    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.324876 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.324859    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 12:07:51.325105 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.325089    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 12:07:51.325325 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.325306    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 12:07:51.325402 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.325316    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 12:07:51.325402 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.325377    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 12:07:51.325402 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.325382    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8q86g\""
Jul 15 12:07:51.325613 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.325600    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 12:07:51.327405 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.327387    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l85fd\""
Jul 15 12:07:51.327466 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.327386    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 12:07:51.327466 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.327387    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 12:07:51.327929 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.327915    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 12:07:51.331938 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.331910    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 12:07:51.372246 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.372226    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gpspg"
Jul 15 12:07:51.383051 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.383036    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gpspg"
Jul 15 12:07:51.418644 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.418616    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 12:07:51.422475 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-socket-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.422565 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b5741b7-0068-4f5e-9db2-7c0c18e7579c-konnectivity-ca\") pod \"konnectivity-agent-49slm\" (UID: \"2b5741b7-0068-4f5e-9db2-7c0c18e7579c\") " pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.422565 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-run-netns\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.422565 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.422693 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422574    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67x9\" (UniqueName: \"kubernetes.io/projected/7df60601-9a45-400a-8937-5f310dc9e70e-kube-api-access-s67x9\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.422693 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-kubelet\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.422693 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422643    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-kubelet\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.422788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-etc-kubernetes\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.422822 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422788    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dac464eb-a63f-470b-883d-93d07e830acb-kubelet-config\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.422868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmghl\" (UniqueName: \"kubernetes.io/projected/69dae2e5-759c-4555-9ac7-b6ced7bb8177-kube-api-access-qmghl\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.422868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422833    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-etc-kubernetes\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.422868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovn-node-metrics-cert\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.423014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dac464eb-a63f-470b-883d-93d07e830acb-kubelet-config\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.423014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/728cb636-c45f-42d8-a03b-b5f54a4fd18d-hosts-file\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.423014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728cb636-c45f-42d8-a03b-b5f54a4fd18d-tmp-dir\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.423014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.422973    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/728cb636-c45f-42d8-a03b-b5f54a4fd18d-hosts-file\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.423014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-os-release\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-hostroot\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-conf-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-hostroot\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423079    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-os-release\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b5741b7-0068-4f5e-9db2-7c0c18e7579c-agent-certs\") pod \"konnectivity-agent-49slm\" (UID: \"2b5741b7-0068-4f5e-9db2-7c0c18e7579c\") " pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423149    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-conf-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423155    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysconfig\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-multus-certs\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423205    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysconfig\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423225    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dac464eb-a63f-470b-883d-93d07e830acb-dbus\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423238    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-multus-certs\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:51.423256 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423264    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/728cb636-c45f-42d8-a03b-b5f54a4fd18d-tmp-dir\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69dae2e5-759c-4555-9ac7-b6ced7bb8177-host\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-sys\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-host\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-system-cni-dir\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423364    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dac464eb-a63f-470b-883d-93d07e830acb-dbus\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-slash\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-var-lib-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423409    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-etc-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423422    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-sys\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423430    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-ovn\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423447    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-run-ovn-kubernetes\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-var-lib-kubelet\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423466    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-host\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.423781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423411    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-system-cni-dir\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423490    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-cni-multus\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-cnibin\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423527    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-var-lib-kubelet\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-cni-multus\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423590    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423600    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-cnibin\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423833    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-device-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423852    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-cni-binary-copy\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423888    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-netns\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-cni-bin\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.424352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7df60601-9a45-400a-8937-5f310dc9e70e-host-slash\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-netns\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423959    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-systemd\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.423988    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-var-lib-cni-bin\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424004    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovnkube-config\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7df60601-9a45-400a-8937-5f310dc9e70e-host-slash\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424029    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-cni-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-cnibin\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-socket-dir-parent\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424130    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-cni-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-k8s-cni-cncf-io\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424166    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-cnibin\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-cni-bin\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424208    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-socket-dir-parent\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-host-run-k8s-cni-cncf-io\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424227    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovnkube-script-lib\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-lib-modules\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424336    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-etc-selinux\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/462ed658-e936-483e-8353-cde6d14feec3-etc-tuned\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424386    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424411    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-os-release\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg5rc\" (UniqueName: \"kubernetes.io/projected/8ee8bd80-e264-413e-84d3-1a40c8c412ca-kube-api-access-lg5rc\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424465    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-lib-modules\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424469    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l6b2\" (UniqueName: \"kubernetes.io/projected/728cb636-c45f-42d8-a03b-b5f54a4fd18d-kube-api-access-6l6b2\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424495    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-cni-binary-copy\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424508    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysctl-d\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424536    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.425650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-systemd-units\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424614    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-modprobe-d\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-kubernetes\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-systemd\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424662    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wq4k\" (UniqueName: \"kubernetes.io/projected/cae9499a-36e9-4a31-aba3-c268211cc3a9-kube-api-access-9wq4k\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/69dae2e5-759c-4555-9ac7-b6ced7bb8177-serviceca\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424710    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6m2x\" (UniqueName: \"kubernetes.io/projected/9a30a548-c8c0-46c0-acc7-e395471e948b-kube-api-access-d6m2x\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424718    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-kubernetes\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424715    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424680    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysctl-d\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424753    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-systemd\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424760    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-env-overrides\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424763    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-modprobe-d\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424787    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzwnc\" (UniqueName: \"kubernetes.io/projected/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-kube-api-access-tzwnc\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysctl-conf\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-run\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-daemon-config\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/462ed658-e936-483e-8353-cde6d14feec3-tmp\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424918    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-run\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.424923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-system-cni-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425486    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-registration-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425046    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-system-cni-dir\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-sys-fs\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425386    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-multus-daemon-config\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425127    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cae9499a-36e9-4a31-aba3-c268211cc3a9-os-release\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-node-log\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.425192    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.425556    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425007    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/462ed658-e936-483e-8353-cde6d14feec3-etc-sysctl-conf\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.426680 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.425621    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:07:51.925604094 +0000 UTC m=+3.025553448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425625    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.425644    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret podName:dac464eb-a63f-470b-883d-93d07e830acb nodeName:}" failed. No retries permitted until 2026-07-15 12:07:51.92563333 +0000 UTC m=+3.025582650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret") pod "global-pull-secret-syncer-h4nz6" (UID: "dac464eb-a63f-470b-883d-93d07e830acb") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-kubelet\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425706    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-log-socket\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425742    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-cni-netd\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5749n\" (UniqueName: \"kubernetes.io/projected/462ed658-e936-483e-8353-cde6d14feec3-kube-api-access-5749n\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj5vz\" (UniqueName: \"kubernetes.io/projected/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-kube-api-access-mj5vz\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.425841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7df60601-9a45-400a-8937-5f310dc9e70e-iptables-alerter-script\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.426046    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cae9499a-36e9-4a31-aba3-c268211cc3a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.427294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.426350    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7df60601-9a45-400a-8937-5f310dc9e70e-iptables-alerter-script\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.428244 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.428227    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/462ed658-e936-483e-8353-cde6d14feec3-tmp\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.428244 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.428234    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/462ed658-e936-483e-8353-cde6d14feec3-etc-tuned\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.430934 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.430918    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:07:51.430934 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.430936    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:07:51.431049 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.430946    2582 projected.go:196] Error preparing data for projected volume kube-api-access-8txhr for pod openshift-network-diagnostics/network-check-target-4qbcd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:51.431049 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.430996    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr podName:9121b055-df43-4988-8bf1-dd0bb1d5ac08 nodeName:}" failed. No retries permitted until 2026-07-15 12:07:51.930985819 +0000 UTC m=+3.030935166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8txhr" (UniqueName: "kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr") pod "network-check-target-4qbcd" (UID: "9121b055-df43-4988-8bf1-dd0bb1d5ac08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:51.431913 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.431879    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67x9\" (UniqueName: \"kubernetes.io/projected/7df60601-9a45-400a-8937-5f310dc9e70e-kube-api-access-s67x9\") pod \"iptables-alerter-rhl8m\" (UID: \"7df60601-9a45-400a-8937-5f310dc9e70e\") " pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.433494 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.433476    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wq4k\" (UniqueName: \"kubernetes.io/projected/cae9499a-36e9-4a31-aba3-c268211cc3a9-kube-api-access-9wq4k\") pod \"multus-additional-cni-plugins-qh85v\" (UID: \"cae9499a-36e9-4a31-aba3-c268211cc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.436059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.436036    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5749n\" (UniqueName: \"kubernetes.io/projected/462ed658-e936-483e-8353-cde6d14feec3-kube-api-access-5749n\") pod \"tuned-8tfkf\" (UID: \"462ed658-e936-483e-8353-cde6d14feec3\") " pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.436445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.436425    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg5rc\" (UniqueName: \"kubernetes.io/projected/8ee8bd80-e264-413e-84d3-1a40c8c412ca-kube-api-access-lg5rc\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:51.437239 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.437214    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj5vz\" (UniqueName: \"kubernetes.io/projected/04a1339e-1e0a-4b7e-8b28-4f5b1db60e56-kube-api-access-mj5vz\") pod \"multus-c2d9c\" (UID: \"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56\") " pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.437354 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.437337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l6b2\" (UniqueName: \"kubernetes.io/projected/728cb636-c45f-42d8-a03b-b5f54a4fd18d-kube-api-access-6l6b2\") pod \"node-resolver-7bkzq\" (UID: \"728cb636-c45f-42d8-a03b-b5f54a4fd18d\") " pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.527176 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527064    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b5741b7-0068-4f5e-9db2-7c0c18e7579c-agent-certs\") pod \"konnectivity-agent-49slm\" (UID: \"2b5741b7-0068-4f5e-9db2-7c0c18e7579c\") " pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.527176 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69dae2e5-759c-4555-9ac7-b6ced7bb8177-host\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.527176 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527139    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-slash\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527195    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69dae2e5-759c-4555-9ac7-b6ced7bb8177-host\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-var-lib-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527248    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-etc-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527257    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-slash\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-ovn\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527283    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-run-ovn-kubernetes\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-device-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-systemd\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovnkube-config\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527371    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-run-ovn-kubernetes\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527383    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-device-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-var-lib-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527339    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-etc-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.527436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527430    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-ovn\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-openvswitch\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527513    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-run-systemd\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527531    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-cni-bin\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527586    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527595    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-cni-bin\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovnkube-script-lib\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527633    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-etc-selinux\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527688    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-systemd-units\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/69dae2e5-759c-4555-9ac7-b6ced7bb8177-serviceca\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527794    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-etc-selinux\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-systemd-units\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6m2x\" (UniqueName: \"kubernetes.io/projected/9a30a548-c8c0-46c0-acc7-e395471e948b-kube-api-access-d6m2x\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527872    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-env-overrides\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwnc\" (UniqueName: \"kubernetes.io/projected/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-kube-api-access-tzwnc\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-registration-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.527977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-sys-fs\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-node-log\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-kubelet\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-registration-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528066    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-sys-fs\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-log-socket\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528095    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovnkube-config\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-cni-netd\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528115    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-log-socket\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.528916 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528135    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-node-log\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528171    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-cni-netd\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-socket-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528171    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-kubelet\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528206    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/69dae2e5-759c-4555-9ac7-b6ced7bb8177-serviceca\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b5741b7-0068-4f5e-9db2-7c0c18e7579c-konnectivity-ca\") pod \"konnectivity-agent-49slm\" (UID: \"2b5741b7-0068-4f5e-9db2-7c0c18e7579c\") " pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-run-netns\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528274    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmghl\" (UniqueName: \"kubernetes.io/projected/69dae2e5-759c-4555-9ac7-b6ced7bb8177-kube-api-access-qmghl\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a30a548-c8c0-46c0-acc7-e395471e948b-socket-dir\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovn-node-metrics-cert\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528330    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-host-run-netns\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528686    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-env-overrides\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528778    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovnkube-script-lib\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.529424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.528890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b5741b7-0068-4f5e-9db2-7c0c18e7579c-konnectivity-ca\") pod \"konnectivity-agent-49slm\" (UID: \"2b5741b7-0068-4f5e-9db2-7c0c18e7579c\") " pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.529996 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.529977    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b5741b7-0068-4f5e-9db2-7c0c18e7579c-agent-certs\") pod \"konnectivity-agent-49slm\" (UID: \"2b5741b7-0068-4f5e-9db2-7c0c18e7579c\") " pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.530638 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.530608    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-ovn-node-metrics-cert\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.544017 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.543985    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmghl\" (UniqueName: \"kubernetes.io/projected/69dae2e5-759c-4555-9ac7-b6ced7bb8177-kube-api-access-qmghl\") pod \"node-ca-sp6l2\" (UID: \"69dae2e5-759c-4555-9ac7-b6ced7bb8177\") " pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.544327 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.544308    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6m2x\" (UniqueName: \"kubernetes.io/projected/9a30a548-c8c0-46c0-acc7-e395471e948b-kube-api-access-d6m2x\") pod \"aws-ebs-csi-driver-node-q8vhp\" (UID: \"9a30a548-c8c0-46c0-acc7-e395471e948b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.544454 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.544437    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzwnc\" (UniqueName: \"kubernetes.io/projected/ec463e22-c92e-4d8a-9f91-a60f8bbc6e24-kube-api-access-tzwnc\") pod \"ovnkube-node-9sx9r\" (UID: \"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.618058 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.618026    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhl8m"
Jul 15 12:07:51.625393 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.625372    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7bkzq"
Jul 15 12:07:51.627826 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.627713    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df60601_9a45_400a_8937_5f310dc9e70e.slice/crio-843c996eae78b60aa98de35d7e3e9b707646709804352097ca91739e32b7a020 WatchSource:0}: Error finding container 843c996eae78b60aa98de35d7e3e9b707646709804352097ca91739e32b7a020: Status 404 returned error can't find the container with id 843c996eae78b60aa98de35d7e3e9b707646709804352097ca91739e32b7a020
Jul 15 12:07:51.632769 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.632744    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8tfkf"
Jul 15 12:07:51.635859 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.635831    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod728cb636_c45f_42d8_a03b_b5f54a4fd18d.slice/crio-af57644485efd6eb35279148c8c25ed585b952fe587c6153c7552abfc021f943 WatchSource:0}: Error finding container af57644485efd6eb35279148c8c25ed585b952fe587c6153c7552abfc021f943: Status 404 returned error can't find the container with id af57644485efd6eb35279148c8c25ed585b952fe587c6153c7552abfc021f943
Jul 15 12:07:51.636986 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.636966    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qh85v"
Jul 15 12:07:51.642579 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.642551    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462ed658_e936_483e_8353_cde6d14feec3.slice/crio-05dfa44c5e5c48dded20da20f47f83e2d4a39d7bc8c5fd84671a3f4a19e41b05 WatchSource:0}: Error finding container 05dfa44c5e5c48dded20da20f47f83e2d4a39d7bc8c5fd84671a3f4a19e41b05: Status 404 returned error can't find the container with id 05dfa44c5e5c48dded20da20f47f83e2d4a39d7bc8c5fd84671a3f4a19e41b05
Jul 15 12:07:51.643333 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.643313    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c2d9c"
Jul 15 12:07:51.649376 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.649354    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp"
Jul 15 12:07:51.649842 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.649816    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae9499a_36e9_4a31_aba3_c268211cc3a9.slice/crio-7885a4fe3b28d803a3d97298bc88292cd9d8e2725060d9a8ef13009fac8d2e0f WatchSource:0}: Error finding container 7885a4fe3b28d803a3d97298bc88292cd9d8e2725060d9a8ef13009fac8d2e0f: Status 404 returned error can't find the container with id 7885a4fe3b28d803a3d97298bc88292cd9d8e2725060d9a8ef13009fac8d2e0f
Jul 15 12:07:51.655546 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.655522    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:07:51.656427 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.656398    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a1339e_1e0a_4b7e_8b28_4f5b1db60e56.slice/crio-4f3b8f084b79201f6e5bf8d76db6b97844a253aacc2f4ecde721cfa0f2eb8003 WatchSource:0}: Error finding container 4f3b8f084b79201f6e5bf8d76db6b97844a253aacc2f4ecde721cfa0f2eb8003: Status 404 returned error can't find the container with id 4f3b8f084b79201f6e5bf8d76db6b97844a253aacc2f4ecde721cfa0f2eb8003
Jul 15 12:07:51.659665 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.659643    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a30a548_c8c0_46c0_acc7_e395471e948b.slice/crio-ad84c083672c903c24a677b36da434325e5ec2969041abd6d1dcb7cbb1c3cf5f WatchSource:0}: Error finding container ad84c083672c903c24a677b36da434325e5ec2969041abd6d1dcb7cbb1c3cf5f: Status 404 returned error can't find the container with id ad84c083672c903c24a677b36da434325e5ec2969041abd6d1dcb7cbb1c3cf5f
Jul 15 12:07:51.660789 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.660769    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:07:51.665283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.665260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sp6l2"
Jul 15 12:07:51.665391 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.665328    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5741b7_0068_4f5e_9db2_7c0c18e7579c.slice/crio-848600ad24aec571eec278616a6401e300f3f69581a441a8e047ffdfdf46b4be WatchSource:0}: Error finding container 848600ad24aec571eec278616a6401e300f3f69581a441a8e047ffdfdf46b4be: Status 404 returned error can't find the container with id 848600ad24aec571eec278616a6401e300f3f69581a441a8e047ffdfdf46b4be
Jul 15 12:07:51.673514 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.673473    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec463e22_c92e_4d8a_9f91_a60f8bbc6e24.slice/crio-6c81f9a900c3c7cc312b883599493d31f360e1df8281cb382134720fad0cf852 WatchSource:0}: Error finding container 6c81f9a900c3c7cc312b883599493d31f360e1df8281cb382134720fad0cf852: Status 404 returned error can't find the container with id 6c81f9a900c3c7cc312b883599493d31f360e1df8281cb382134720fad0cf852
Jul 15 12:07:51.679128 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:07:51.679101    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69dae2e5_759c_4555_9ac7_b6ced7bb8177.slice/crio-13e5ade69fde7c49d337f84db6ed8ac630b3596d963698b00239957e1f3d6b7e WatchSource:0}: Error finding container 13e5ade69fde7c49d337f84db6ed8ac630b3596d963698b00239957e1f3d6b7e: Status 404 returned error can't find the container with id 13e5ade69fde7c49d337f84db6ed8ac630b3596d963698b00239957e1f3d6b7e
Jul 15 12:07:51.931680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.931593    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:51.931680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.931650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:51.931910 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:51.931693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:51.931910 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931806    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:51.931910 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931860    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:07:52.931845201 +0000 UTC m=+4.031794545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:51.931910 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931881    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:51.931910 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931804    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:07:51.932201 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931921    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:07:51.932201 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931937    2582 projected.go:196] Error preparing data for projected volume kube-api-access-8txhr for pod openshift-network-diagnostics/network-check-target-4qbcd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:51.932201 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931944    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret podName:dac464eb-a63f-470b-883d-93d07e830acb nodeName:}" failed. No retries permitted until 2026-07-15 12:07:52.931925878 +0000 UTC m=+4.031875213 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret") pod "global-pull-secret-syncer-h4nz6" (UID: "dac464eb-a63f-470b-883d-93d07e830acb") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:51.932201 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:51.931992    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr podName:9121b055-df43-4988-8bf1-dd0bb1d5ac08 nodeName:}" failed. No retries permitted until 2026-07-15 12:07:52.931973917 +0000 UTC m=+4.031923271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8txhr" (UniqueName: "kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr") pod "network-check-target-4qbcd" (UID: "9121b055-df43-4988-8bf1-dd0bb1d5ac08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:52.005685 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.005649    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:07:52.061462 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.061399    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sp6l2" event={"ID":"69dae2e5-759c-4555-9ac7-b6ced7bb8177","Type":"ContainerStarted","Data":"13e5ade69fde7c49d337f84db6ed8ac630b3596d963698b00239957e1f3d6b7e"}
Jul 15 12:07:52.062729 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.062696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"6c81f9a900c3c7cc312b883599493d31f360e1df8281cb382134720fad0cf852"}
Jul 15 12:07:52.063859 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.063832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49slm" event={"ID":"2b5741b7-0068-4f5e-9db2-7c0c18e7579c","Type":"ContainerStarted","Data":"848600ad24aec571eec278616a6401e300f3f69581a441a8e047ffdfdf46b4be"}
Jul 15 12:07:52.065110 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.065071    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp" event={"ID":"9a30a548-c8c0-46c0-acc7-e395471e948b","Type":"ContainerStarted","Data":"ad84c083672c903c24a677b36da434325e5ec2969041abd6d1dcb7cbb1c3cf5f"}
Jul 15 12:07:52.066238 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.066216    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c2d9c" event={"ID":"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56","Type":"ContainerStarted","Data":"4f3b8f084b79201f6e5bf8d76db6b97844a253aacc2f4ecde721cfa0f2eb8003"}
Jul 15 12:07:52.067423 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.067402    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8tfkf" event={"ID":"462ed658-e936-483e-8353-cde6d14feec3","Type":"ContainerStarted","Data":"05dfa44c5e5c48dded20da20f47f83e2d4a39d7bc8c5fd84671a3f4a19e41b05"}
Jul 15 12:07:52.068568 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.068538    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7bkzq" event={"ID":"728cb636-c45f-42d8-a03b-b5f54a4fd18d","Type":"ContainerStarted","Data":"af57644485efd6eb35279148c8c25ed585b952fe587c6153c7552abfc021f943"}
Jul 15 12:07:52.069722 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.069701    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" event={"ID":"3aa7a67259a3462e5ba6775b001f3248","Type":"ContainerStarted","Data":"693175539b9e27a125044eb6139a9ad629fb61c0e87094ae7d5d082f6f867fa8"}
Jul 15 12:07:52.070846 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.070826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerStarted","Data":"7885a4fe3b28d803a3d97298bc88292cd9d8e2725060d9a8ef13009fac8d2e0f"}
Jul 15 12:07:52.071977 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.071954    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhl8m" event={"ID":"7df60601-9a45-400a-8937-5f310dc9e70e","Type":"ContainerStarted","Data":"843c996eae78b60aa98de35d7e3e9b707646709804352097ca91739e32b7a020"}
Jul 15 12:07:52.386204 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.386155    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:02:51 +0000 UTC" deadline="2028-03-23 10:38:40.200577296 +0000 UTC"
Jul 15 12:07:52.386204 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.386202    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14806h30m47.814377807s"
Jul 15 12:07:52.941862 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.941420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:52.941862 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.941725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:52.941862 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:52.941784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:52.942136 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942009    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:07:52.942136 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942034    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:07:52.942136 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942049    2582 projected.go:196] Error preparing data for projected volume kube-api-access-8txhr for pod openshift-network-diagnostics/network-check-target-4qbcd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:52.942136 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942121    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr podName:9121b055-df43-4988-8bf1-dd0bb1d5ac08 nodeName:}" failed. No retries permitted until 2026-07-15 12:07:54.942101192 +0000 UTC m=+6.042050513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8txhr" (UniqueName: "kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr") pod "network-check-target-4qbcd" (UID: "9121b055-df43-4988-8bf1-dd0bb1d5ac08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:52.942584 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942563    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:52.942659 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942621    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret podName:dac464eb-a63f-470b-883d-93d07e830acb nodeName:}" failed. No retries permitted until 2026-07-15 12:07:54.942606101 +0000 UTC m=+6.042555423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret") pod "global-pull-secret-syncer-h4nz6" (UID: "dac464eb-a63f-470b-883d-93d07e830acb") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:52.942732 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942684    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:52.942732 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:52.942716    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:07:54.942704522 +0000 UTC m=+6.042653849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:53.058412 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:53.056825    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:53.058412 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:53.056975    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:07:53.058412 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:53.057145    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:53.058412 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:53.057284    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:07:53.058412 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:53.057357    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:53.058412 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:53.057435    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:07:53.083190 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:53.083148    2582 generic.go:356] "Generic (PLEG): container finished" podID="3aa7a67259a3462e5ba6775b001f3248" containerID="02f59870b88ce54c80db836b14664e72e78c887afeae8e4e2ebc2bdd8fddba6d" exitCode=0
Jul 15 12:07:53.083349 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:53.083215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" event={"ID":"3aa7a67259a3462e5ba6775b001f3248","Type":"ContainerDied","Data":"02f59870b88ce54c80db836b14664e72e78c887afeae8e4e2ebc2bdd8fddba6d"}
Jul 15 12:07:53.387330 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:53.387212    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:02:51 +0000 UTC" deadline="2028-04-11 18:03:31.81965872 +0000 UTC"
Jul 15 12:07:53.387330 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:53.387282    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15269h55m38.432381032s"
Jul 15 12:07:54.090367 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:54.090326    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" event={"ID":"3aa7a67259a3462e5ba6775b001f3248","Type":"ContainerStarted","Data":"d0bd73f72c9682b45ea1e1c1303a165e8c40b324a9f7e0867e179eea10e09bf6"}
Jul 15 12:07:54.099421 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:54.099390    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" event={"ID":"b45ab77f7adac80ae4f46c80d27c83b4","Type":"ContainerStarted","Data":"44731c828af4c91dd16d1199dfea9ee7b6a8a27e97af145c6e865204126d791a"}
Jul 15 12:07:54.122927 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:54.121733    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" podStartSLOduration=3.121715141 podStartE2EDuration="3.121715141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:07:54.121446453 +0000 UTC m=+5.221395796" watchObservedRunningTime="2026-07-15 12:07:54.121715141 +0000 UTC m=+5.221664484"
Jul 15 12:07:54.963649 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:54.963586    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:54.963649 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:54.963655    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:54.963705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963747    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963784    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963810    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:07:58.963789749 +0000 UTC m=+10.063739090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963829    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret podName:dac464eb-a63f-470b-883d-93d07e830acb nodeName:}" failed. No retries permitted until 2026-07-15 12:07:58.963818728 +0000 UTC m=+10.063768056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret") pod "global-pull-secret-syncer-h4nz6" (UID: "dac464eb-a63f-470b-883d-93d07e830acb") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963886    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963902    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963913    2582 projected.go:196] Error preparing data for projected volume kube-api-access-8txhr for pod openshift-network-diagnostics/network-check-target-4qbcd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:54.964420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:54.963950    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr podName:9121b055-df43-4988-8bf1-dd0bb1d5ac08 nodeName:}" failed. No retries permitted until 2026-07-15 12:07:58.963938031 +0000 UTC m=+10.063887353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8txhr" (UniqueName: "kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr") pod "network-check-target-4qbcd" (UID: "9121b055-df43-4988-8bf1-dd0bb1d5ac08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:55.057296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:55.056510    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:55.057296 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:55.056642    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:07:55.057296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:55.056723    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:55.057296 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:55.056815    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:07:55.057296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:55.056870    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:55.057296 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:55.056932    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:07:57.056500 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:57.056460    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:57.056500 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:57.056481    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:57.057026 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:57.056510    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:57.057026 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:57.056597    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:07:57.057026 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:57.056636    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:07:57.057026 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:57.056701    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:59.001071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:59.001156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:59.001204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001238    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001318    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001326    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret podName:dac464eb-a63f-470b-883d-93d07e830acb nodeName:}" failed. No retries permitted until 2026-07-15 12:08:07.001297519 +0000 UTC m=+18.101246841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret") pod "global-pull-secret-syncer-h4nz6" (UID: "dac464eb-a63f-470b-883d-93d07e830acb") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001389    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001410    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:07:59.001418 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001422    2582 projected.go:196] Error preparing data for projected volume kube-api-access-8txhr for pod openshift-network-diagnostics/network-check-target-4qbcd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:59.002309 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001395    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:08:07.001377146 +0000 UTC m=+18.101326477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:07:59.002309 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.001473    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr podName:9121b055-df43-4988-8bf1-dd0bb1d5ac08 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:07.001461511 +0000 UTC m=+18.101410829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8txhr" (UniqueName: "kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr") pod "network-check-target-4qbcd" (UID: "9121b055-df43-4988-8bf1-dd0bb1d5ac08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:07:59.057283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:59.057249    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:07:59.057448 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:59.057253    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:07:59.057448 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.057370    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:07:59.057448 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:07:59.057383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:07:59.057615 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.057479    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:07:59.057615 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:07:59.057575    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:01.056738 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:01.056699    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:01.057233 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:01.056699    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:01.057233 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:01.056835    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:01.057233 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:01.056699    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:01.057233 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:01.056937    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:01.057233 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:01.057027    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:03.057285 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:03.057201    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:03.057785 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:03.057213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:03.057785 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:03.057353    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:03.057785 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:03.057219    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:03.057785 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:03.057423    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:03.057785 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:03.057513    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:05.057112 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:05.057053    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:05.057575 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:05.057116    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:05.057575 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:05.057116    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:05.057575 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:05.057249    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:05.057575 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:05.057353    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:05.057575 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:05.057468    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:07.056936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:07.056827    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:07.056936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:07.056860    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:07.056936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:07.056861    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:07.057470 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.056973    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:07.057470 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.057030    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:07.057470 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.057144    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:07.057981 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:07.057951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:07.058116 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:07.058001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:07.058116 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:07.058028    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:07.058116 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058080    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:08:07.058291 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058121    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:08:07.058291 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058133    2582 projected.go:196] Error preparing data for projected volume kube-api-access-8txhr for pod openshift-network-diagnostics/network-check-target-4qbcd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:08:07.058291 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058167    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:08:07.058291 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058171    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:08:07.058291 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058173    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr podName:9121b055-df43-4988-8bf1-dd0bb1d5ac08 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:23.058158072 +0000 UTC m=+34.158107389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8txhr" (UniqueName: "kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr") pod "network-check-target-4qbcd" (UID: "9121b055-df43-4988-8bf1-dd0bb1d5ac08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:08:07.058291 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058256    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret podName:dac464eb-a63f-470b-883d-93d07e830acb nodeName:}" failed. No retries permitted until 2026-07-15 12:08:23.058238639 +0000 UTC m=+34.158187962 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret") pod "global-pull-secret-syncer-h4nz6" (UID: "dac464eb-a63f-470b-883d-93d07e830acb") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:08:07.058291 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:07.058273    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:08:23.05826281 +0000 UTC m=+34.158212133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:08:09.056681 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:09.056641    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:09.056681 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:09.056674    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:09.057250 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:09.056695    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:09.057250 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:09.056779    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:09.057250 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:09.056846    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:09.057250 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:09.056935    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:11.056648 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:11.056618    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:11.056648 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:11.056638    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:11.057028 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:11.056618    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:11.057028 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:11.056729    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:11.057028 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:11.056783    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:11.057028 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:11.056862    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:12.141476 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.141193    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sp6l2" event={"ID":"69dae2e5-759c-4555-9ac7-b6ced7bb8177","Type":"ContainerStarted","Data":"fe5f76759d54a3350b731d4b47af2497df0a9eb7c813a820649e0d2d98851a89"}
Jul 15 12:08:12.143693 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.143661    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"980235c08983332f8dcd401ea416f0adef3e5903ac1ca1b1c05991a4a3209b20"}
Jul 15 12:08:12.145023 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.144980    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49slm" event={"ID":"2b5741b7-0068-4f5e-9db2-7c0c18e7579c","Type":"ContainerStarted","Data":"6ad72a934d5abcfa0e795dfcf858e6b35a791c4ac32dab43a563c09311ae91b4"}
Jul 15 12:08:12.146499 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.146467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp" event={"ID":"9a30a548-c8c0-46c0-acc7-e395471e948b","Type":"ContainerStarted","Data":"57b1c5c54652dabc436ba275eb1c8c6e4eec62e6a6d64733528e03445f714e63"}
Jul 15 12:08:12.148495 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.148470    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c2d9c" event={"ID":"04a1339e-1e0a-4b7e-8b28-4f5b1db60e56","Type":"ContainerStarted","Data":"f3d5baddfd1af995675a4409cd34979eba755ee55857c3ed17e0614580ef0456"}
Jul 15 12:08:12.150072 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.150047    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8tfkf" event={"ID":"462ed658-e936-483e-8353-cde6d14feec3","Type":"ContainerStarted","Data":"8b8622f6bb2b2f73e266ef51e16b46b1b6eff616985da615b7f902ce9f83b9e3"}
Jul 15 12:08:12.151527 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.151508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7bkzq" event={"ID":"728cb636-c45f-42d8-a03b-b5f54a4fd18d","Type":"ContainerStarted","Data":"b8ff57c32686f0d2d078964da74733dabeffdff782f8f989fd9f0379da0db41c"}
Jul 15 12:08:12.152990 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.152966    2582 generic.go:356] "Generic (PLEG): container finished" podID="cae9499a-36e9-4a31-aba3-c268211cc3a9" containerID="f1958e2cdfd73c229f4ff8b3a157aa02838e38e9aa66ddf40b1354ba5706d91c" exitCode=0
Jul 15 12:08:12.153074 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.153043    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerDied","Data":"f1958e2cdfd73c229f4ff8b3a157aa02838e38e9aa66ddf40b1354ba5706d91c"}
Jul 15 12:08:12.154571 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.154546    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" event={"ID":"b45ab77f7adac80ae4f46c80d27c83b4","Type":"ContainerStarted","Data":"a17728e5ef8d9a4074fcfa41721504ec4279f348023306f0bfd0a92d7c98fada"}
Jul 15 12:08:12.158496 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.158454    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sp6l2" podStartSLOduration=2.358612515 podStartE2EDuration="22.15844298s" totalImagesPullingTime="19.799830465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:12.158022572 +0000 UTC m=+23.257971911" watchObservedRunningTime="2026-07-15 12:08:12.15844298 +0000 UTC m=+23.258392319"
Jul 15 12:08:12.173830 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.173778    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7bkzq" podStartSLOduration=1.992528975 podStartE2EDuration="22.173761436s" totalImagesPullingTime="20.181232461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:12.173012435 +0000 UTC m=+23.272961774" watchObservedRunningTime="2026-07-15 12:08:12.173761436 +0000 UTC m=+23.273710774"
Jul 15 12:08:12.220986 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.220936    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8tfkf" podStartSLOduration=2.047934008 podStartE2EDuration="22.220919619s" totalImagesPullingTime="20.172985611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:12.196699119 +0000 UTC m=+23.296648458" watchObservedRunningTime="2026-07-15 12:08:12.220919619 +0000 UTC m=+23.320868958"
Jul 15 12:08:12.247159 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.247114    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c2d9c" podStartSLOduration=2.083746214 podStartE2EDuration="22.247074806s" totalImagesPullingTime="20.163328592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:12.24540729 +0000 UTC m=+23.345356628" watchObservedRunningTime="2026-07-15 12:08:12.247074806 +0000 UTC m=+23.347024145"
Jul 15 12:08:12.261405 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:12.261365    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-49slm" podStartSLOduration=2.447595467 podStartE2EDuration="22.261348603s" totalImagesPullingTime="19.813753136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:12.260818111 +0000 UTC m=+23.360767449" watchObservedRunningTime="2026-07-15 12:08:12.261348603 +0000 UTC m=+23.361297941"
Jul 15 12:08:13.057474 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.057296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:13.057565 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.057295    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:13.057605 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:13.057572    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:13.057644 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.057297    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:13.057644 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:13.057625    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:13.057708 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:13.057695    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:13.159760 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.159736    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:08:13.160183 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.160158    2582 generic.go:356] "Generic (PLEG): container finished" podID="ec463e22-c92e-4d8a-9f91-a60f8bbc6e24" containerID="c0b8c92ced24ae0ee3582e724f0eb8722357789e7d401755dd3b31e90a82df4d" exitCode=1
Jul 15 12:08:13.160253 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.160228    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"5376b711e1da15ed9490db693e3207197e1b989a853b14df2721f79fd477d026"}
Jul 15 12:08:13.160331 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.160315    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"6148dda3f0280d242fa20cf811306e2e8cfcc5c9b4c640f34552a322f7956880"}
Jul 15 12:08:13.160388 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.160368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"1a2eee82fa8b61d86c87a60ccd87d84bbca508fbfc331b28bbcced7195445fc5"}
Jul 15 12:08:13.160437 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.160417    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"caa93cce36b4845006e3beb4ad49cbfdcf98287468b636714162a403d8307659"}
Jul 15 12:08:13.160481 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.160466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerDied","Data":"c0b8c92ced24ae0ee3582e724f0eb8722357789e7d401755dd3b31e90a82df4d"}
Jul 15 12:08:13.162381 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.162356    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhl8m" event={"ID":"7df60601-9a45-400a-8937-5f310dc9e70e","Type":"ContainerStarted","Data":"aa5d80bca2a4ab9a9042bf1eef7c6111f0be374237ec3ab72d805b712adebc37"}
Jul 15 12:08:13.178571 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.178520    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" podStartSLOduration=22.178502474 podStartE2EDuration="22.178502474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:12.276034452 +0000 UTC m=+23.375983792" watchObservedRunningTime="2026-07-15 12:08:13.178502474 +0000 UTC m=+24.278451816"
Jul 15 12:08:13.178851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.178832    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rhl8m" podStartSLOduration=2.991805668 podStartE2EDuration="23.178828362s" totalImagesPullingTime="20.187022694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:13.178393613 +0000 UTC m=+24.278342954" watchObservedRunningTime="2026-07-15 12:08:13.178828362 +0000 UTC m=+24.278777700"
Jul 15 12:08:13.237491 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.237455    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 12:08:13.931414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.931007    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T12:08:13.237475857Z","UUID":"d088bb40-36fd-46bb-b97e-4cb06a63034e","Handler":null,"Name":"","Endpoint":""}
Jul 15 12:08:13.932678 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.932649    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 12:08:13.932802 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:13.932688    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 12:08:14.167960 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:14.167917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp" event={"ID":"9a30a548-c8c0-46c0-acc7-e395471e948b","Type":"ContainerStarted","Data":"1f833a5c85a045a0924b3b4e1731285feb9bf77dd060107529cf0d2b634bb0c5"}
Jul 15 12:08:15.056458 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:15.056382    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:15.056619 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:15.056382    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:15.056619 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:15.056494    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:15.056619 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:15.056592    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:15.056619 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:15.056383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:15.056825 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:15.056708    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:15.172335 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:15.172288    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp" event={"ID":"9a30a548-c8c0-46c0-acc7-e395471e948b","Type":"ContainerStarted","Data":"0de50483f05914cca652882e03fce46085e91ab14f28af2d6fb4534228685507"}
Jul 15 12:08:15.175633 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:15.175609    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:08:15.176046 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:15.176014    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"e7e3ad2c2715bc1679ddc4c366c7f639277b28a984a6fa8f59e1208e4740b5a5"}
Jul 15 12:08:15.193076 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:15.193035    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q8vhp" podStartSLOduration=3.143870364 podStartE2EDuration="25.193018526s" totalImagesPullingTime="22.049148162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:15.19233363 +0000 UTC m=+26.292282970" watchObservedRunningTime="2026-07-15 12:08:15.193018526 +0000 UTC m=+26.292967866"
Jul 15 12:08:17.019265 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:17.019227    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:08:17.020012 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:17.019996    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:08:17.057245 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:17.057224    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:17.057386 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:17.057340    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:17.057386 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:17.057337    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:17.057486 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:17.057418    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:17.057486 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:17.057350    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:17.057586 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:17.057487    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:17.181077 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:17.180987    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:08:17.181637 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:17.181618    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-49slm"
Jul 15 12:08:18.186058 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:18.185891    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:08:18.186558 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:18.186418    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"b3d3dc7b81c9d5ddafe8727fa158e507394e29a740620c03b1823499459279ac"}
Jul 15 12:08:18.186672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:18.186650    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:08:18.186852 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:18.186832    2582 scope.go:117] "RemoveContainer" containerID="c0b8c92ced24ae0ee3582e724f0eb8722357789e7d401755dd3b31e90a82df4d"
Jul 15 12:08:18.188351 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:18.188329    2582 generic.go:356] "Generic (PLEG): container finished" podID="cae9499a-36e9-4a31-aba3-c268211cc3a9" containerID="ded9a9ff57439e6752eea63ca40ff184c4d622481e42f25affee73c571aa6679" exitCode=0
Jul 15 12:08:18.188444 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:18.188386    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerDied","Data":"ded9a9ff57439e6752eea63ca40ff184c4d622481e42f25affee73c571aa6679"}
Jul 15 12:08:18.203242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:18.203195    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:08:19.057435 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.056721    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:19.057435 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:19.056848    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:19.057435 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.056915    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:19.057435 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:19.056982    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:19.057435 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.057027    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:19.057435 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:19.057087    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:19.194080 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.194052    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:08:19.194565 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.194540    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" event={"ID":"ec463e22-c92e-4d8a-9f91-a60f8bbc6e24","Type":"ContainerStarted","Data":"f13b4a96345dbb55fc17b320320ad869699497936cd9862d70fccb2cfee77380"}
Jul 15 12:08:19.194720 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.194701    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:08:19.194790 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.194773    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:08:19.209817 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.209795    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:08:19.227386 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.227340    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r" podStartSLOduration=8.918639447 podStartE2EDuration="29.22732319s" totalImagesPullingTime="20.308683743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:19.224032444 +0000 UTC m=+30.323981794" watchObservedRunningTime="2026-07-15 12:08:19.22732319 +0000 UTC m=+30.327272528"
Jul 15 12:08:19.707887 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.707650    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h4nz6"]
Jul 15 12:08:19.708021 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.707923    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:19.708064 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:19.708013    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:19.708182 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.708163    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4qbcd"]
Jul 15 12:08:19.708250 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.708240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:19.708341 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:19.708324    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:19.708632 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.708615    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jcw5"]
Jul 15 12:08:19.708700 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:19.708690    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:19.708776 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:19.708761    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:20.198617 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:20.198526    2582 generic.go:356] "Generic (PLEG): container finished" podID="cae9499a-36e9-4a31-aba3-c268211cc3a9" containerID="ea227cd7de85d98cfc1abc96347e3be945376bf27cd0a4faaead015a93bb9255" exitCode=0
Jul 15 12:08:20.198617 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:20.198604    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerDied","Data":"ea227cd7de85d98cfc1abc96347e3be945376bf27cd0a4faaead015a93bb9255"}
Jul 15 12:08:21.056670 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:21.056642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:21.056811 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:21.056753    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:22.057077 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:22.056989    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:22.057810 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:22.057113    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:22.057810 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:22.057156    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:22.057810 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:22.057268    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:22.205381 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:22.205343    2582 generic.go:356] "Generic (PLEG): container finished" podID="cae9499a-36e9-4a31-aba3-c268211cc3a9" containerID="8fc5e1e715e8dc6d4324630ca1104b0e745876c4d5d8d9e7def321ad3ddbf10a" exitCode=0
Jul 15 12:08:22.205537 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:22.205409    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerDied","Data":"8fc5e1e715e8dc6d4324630ca1104b0e745876c4d5d8d9e7def321ad3ddbf10a"}
Jul 15 12:08:23.057272 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:23.057240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:23.057702 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.057352    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:23.077355 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:23.077322    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:23.077536 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:23.077372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:23.077536 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:23.077412    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:23.077536 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077486    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:08:23.077536 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077534    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:08:23.077720 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077552    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:08:55.077535465 +0000 UTC m=+66.177484807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:08:23.077720 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077556    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:08:23.077720 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077486    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:08:23.077720 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077569    2582 projected.go:196] Error preparing data for projected volume kube-api-access-8txhr for pod openshift-network-diagnostics/network-check-target-4qbcd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:08:23.077720 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077595    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret podName:dac464eb-a63f-470b-883d-93d07e830acb nodeName:}" failed. No retries permitted until 2026-07-15 12:08:55.077584945 +0000 UTC m=+66.177534325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret") pod "global-pull-secret-syncer-h4nz6" (UID: "dac464eb-a63f-470b-883d-93d07e830acb") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:08:23.077720 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:23.077618    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr podName:9121b055-df43-4988-8bf1-dd0bb1d5ac08 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:55.077602738 +0000 UTC m=+66.177552068 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8txhr" (UniqueName: "kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr") pod "network-check-target-4qbcd" (UID: "9121b055-df43-4988-8bf1-dd0bb1d5ac08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:08:24.057554 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:24.057349    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:24.058026 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:24.057397    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:24.058026 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:24.057658    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qbcd" podUID="9121b055-df43-4988-8bf1-dd0bb1d5ac08"
Jul 15 12:08:24.058026 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:24.057738    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4nz6" podUID="dac464eb-a63f-470b-883d-93d07e830acb"
Jul 15 12:08:25.056962 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.056921    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:25.057154 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:25.057042    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jcw5" podUID="8ee8bd80-e264-413e-84d3-1a40c8c412ca"
Jul 15 12:08:25.546358 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.546328    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeReady"
Jul 15 12:08:25.546796 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.546499    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 12:08:25.584828 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.584787    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-98b64dc76-n5wmb"]
Jul 15 12:08:25.587410 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.587381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.589950 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.589859    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 12:08:25.589950 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.589939    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 12:08:25.590175 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.589981    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 12:08:25.590175 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.589981    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p8szp\""
Jul 15 12:08:25.595349 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.595315    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 12:08:25.610300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.610214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-98b64dc76-n5wmb"]
Jul 15 12:08:25.610969 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.610944    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7kp2x"]
Jul 15 12:08:25.613405 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.613376    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hqwzl"]
Jul 15 12:08:25.613568 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.613540    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.615109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.615077    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:25.615666 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.615645    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 12:08:25.615752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.615711    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cbvbt\""
Jul 15 12:08:25.615961 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.615944    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 12:08:25.617383 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.617361    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lwhrs\""
Jul 15 12:08:25.617611 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.617591    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 12:08:25.618035 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.618018    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 12:08:25.618705 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.618688    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 12:08:25.631936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.631913    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7kp2x"]
Jul 15 12:08:25.632629 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.632578    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqwzl"]
Jul 15 12:08:25.699730 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.699730 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-bound-sa-token\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.699945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34d1fcc1-504d-4d13-bd09-50803e3981df-config-volume\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.699945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d1fcc1-504d-4d13-bd09-50803e3981df-tmp-dir\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.699945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699804    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.699945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699887    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-installation-pull-secrets\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.700141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699963    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82q4m\" (UniqueName: \"kubernetes.io/projected/34d1fcc1-504d-4d13-bd09-50803e3981df-kube-api-access-82q4m\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.700141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.699990    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-image-registry-private-configuration\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.700141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.700020    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-certificates\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.700141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.700045    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jm5n\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-kube-api-access-2jm5n\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.700141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.700085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7980fcb6-1d65-4f03-b503-3b2bea12e387-ca-trust-extracted\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.700141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.700117    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-trusted-ca\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.700388 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.700166    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6aefad-a945-40b1-8746-e45c0084de59-cert\") pod \"ingress-canary-hqwzl\" (UID: \"fc6aefad-a945-40b1-8746-e45c0084de59\") " pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:25.700388 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.700186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q47c\" (UniqueName: \"kubernetes.io/projected/fc6aefad-a945-40b1-8746-e45c0084de59-kube-api-access-5q47c\") pod \"ingress-canary-hqwzl\" (UID: \"fc6aefad-a945-40b1-8746-e45c0084de59\") " pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:25.800941 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.800902    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82q4m\" (UniqueName: \"kubernetes.io/projected/34d1fcc1-504d-4d13-bd09-50803e3981df-kube-api-access-82q4m\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.801150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.800952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-image-registry-private-configuration\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.800980    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-certificates\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jm5n\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-kube-api-access-2jm5n\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801051    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7980fcb6-1d65-4f03-b503-3b2bea12e387-ca-trust-extracted\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-trusted-ca\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6aefad-a945-40b1-8746-e45c0084de59-cert\") pod \"ingress-canary-hqwzl\" (UID: \"fc6aefad-a945-40b1-8746-e45c0084de59\") " pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:25.801369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q47c\" (UniqueName: \"kubernetes.io/projected/fc6aefad-a945-40b1-8746-e45c0084de59-kube-api-access-5q47c\") pod \"ingress-canary-hqwzl\" (UID: \"fc6aefad-a945-40b1-8746-e45c0084de59\") " pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:25.801369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801289    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.801369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-bound-sa-token\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801347    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34d1fcc1-504d-4d13-bd09-50803e3981df-config-volume\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.801657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801374    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d1fcc1-504d-4d13-bd09-50803e3981df-tmp-dir\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.801657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-installation-pull-secrets\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801657 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:25.801439    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:08:25.801657 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:25.801490    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls podName:34d1fcc1-504d-4d13-bd09-50803e3981df nodeName:}" failed. No retries permitted until 2026-07-15 12:08:26.301474832 +0000 UTC m=+37.401424180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls") pod "dns-default-7kp2x" (UID: "34d1fcc1-504d-4d13-bd09-50803e3981df") : secret "dns-default-metrics-tls" not found
Jul 15 12:08:25.801657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801517    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7980fcb6-1d65-4f03-b503-3b2bea12e387-ca-trust-extracted\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.801948 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.801821    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-certificates\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.802079 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.802050    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34d1fcc1-504d-4d13-bd09-50803e3981df-config-volume\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.802176 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.802154    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34d1fcc1-504d-4d13-bd09-50803e3981df-tmp-dir\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.802242 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:25.802178    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:08:25.802242 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:25.802195    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98b64dc76-n5wmb: secret "image-registry-tls" not found
Jul 15 12:08:25.802328 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:25.802242    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls podName:7980fcb6-1d65-4f03-b503-3b2bea12e387 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:26.302225379 +0000 UTC m=+37.402174701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls") pod "image-registry-98b64dc76-n5wmb" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387") : secret "image-registry-tls" not found
Jul 15 12:08:25.802328 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.802161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-trusted-ca\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.806391 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.806369    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-image-registry-private-configuration\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.806512 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.806396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-installation-pull-secrets\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.806512 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.806407    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6aefad-a945-40b1-8746-e45c0084de59-cert\") pod \"ingress-canary-hqwzl\" (UID: \"fc6aefad-a945-40b1-8746-e45c0084de59\") " pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:25.810188 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.810162    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-bound-sa-token\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.810303 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.810222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82q4m\" (UniqueName: \"kubernetes.io/projected/34d1fcc1-504d-4d13-bd09-50803e3981df-kube-api-access-82q4m\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:25.810504 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.810483    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q47c\" (UniqueName: \"kubernetes.io/projected/fc6aefad-a945-40b1-8746-e45c0084de59-kube-api-access-5q47c\") pod \"ingress-canary-hqwzl\" (UID: \"fc6aefad-a945-40b1-8746-e45c0084de59\") " pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:25.810980 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.810957    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jm5n\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-kube-api-access-2jm5n\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:25.929999 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:25.929910    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqwzl"
Jul 15 12:08:26.056538 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.056500    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:26.056722 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.056500    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:26.059436 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.059415    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 12:08:26.059593 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.059569    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 12:08:26.059709 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.059691    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-674f4\""
Jul 15 12:08:26.059778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.059723    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 12:08:26.304919 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.304890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:26.305019 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.304934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:26.305087 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:26.305062    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:08:26.305227 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:26.305136    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:08:26.305227 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:26.305157    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98b64dc76-n5wmb: secret "image-registry-tls" not found
Jul 15 12:08:26.305227 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:26.305148    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls podName:34d1fcc1-504d-4d13-bd09-50803e3981df nodeName:}" failed. No retries permitted until 2026-07-15 12:08:27.305130944 +0000 UTC m=+38.405080273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls") pod "dns-default-7kp2x" (UID: "34d1fcc1-504d-4d13-bd09-50803e3981df") : secret "dns-default-metrics-tls" not found
Jul 15 12:08:26.305227 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:26.305227    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls podName:7980fcb6-1d65-4f03-b503-3b2bea12e387 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:27.305208412 +0000 UTC m=+38.405157742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls") pod "image-registry-98b64dc76-n5wmb" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387") : secret "image-registry-tls" not found
Jul 15 12:08:26.401186 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.401151    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqwzl"]
Jul 15 12:08:26.406620 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:08:26.406591    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6aefad_a945_40b1_8746_e45c0084de59.slice/crio-8146004ea4ab6097702b2b3f852be1a24c4ffaeac3567eb5f4e06509ff6912fd WatchSource:0}: Error finding container 8146004ea4ab6097702b2b3f852be1a24c4ffaeac3567eb5f4e06509ff6912fd: Status 404 returned error can't find the container with id 8146004ea4ab6097702b2b3f852be1a24c4ffaeac3567eb5f4e06509ff6912fd
Jul 15 12:08:26.789445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.789259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:26.883783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.883744    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:26.954538 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:26.954503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:27.057055 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.056977    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:27.059716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.059693    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 12:08:27.059855 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.059802    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s8wbv\""
Jul 15 12:08:27.222420 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.222381    2582 generic.go:356] "Generic (PLEG): container finished" podID="cae9499a-36e9-4a31-aba3-c268211cc3a9" containerID="f79b9f35adcdcb6be4adc27a8fc81a06783f8099b9921610ca6ba1732ba55a06" exitCode=0
Jul 15 12:08:27.222572 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.222475    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerDied","Data":"f79b9f35adcdcb6be4adc27a8fc81a06783f8099b9921610ca6ba1732ba55a06"}
Jul 15 12:08:27.223751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.223720    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqwzl" event={"ID":"fc6aefad-a945-40b1-8746-e45c0084de59","Type":"ContainerStarted","Data":"8146004ea4ab6097702b2b3f852be1a24c4ffaeac3567eb5f4e06509ff6912fd"}
Jul 15 12:08:27.311414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.311381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:27.311518 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:27.311450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:27.311583 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:27.311564    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:08:27.311642 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:27.311631    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls podName:34d1fcc1-504d-4d13-bd09-50803e3981df nodeName:}" failed. No retries permitted until 2026-07-15 12:08:29.311609982 +0000 UTC m=+40.411559307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls") pod "dns-default-7kp2x" (UID: "34d1fcc1-504d-4d13-bd09-50803e3981df") : secret "dns-default-metrics-tls" not found
Jul 15 12:08:27.311963 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:27.311942    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:08:27.311963 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:27.311962    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98b64dc76-n5wmb: secret "image-registry-tls" not found
Jul 15 12:08:27.312210 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:27.312009    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls podName:7980fcb6-1d65-4f03-b503-3b2bea12e387 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:29.311994152 +0000 UTC m=+40.411943476 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls") pod "image-registry-98b64dc76-n5wmb" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387") : secret "image-registry-tls" not found
Jul 15 12:08:28.230606 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:28.230553    2582 generic.go:356] "Generic (PLEG): container finished" podID="cae9499a-36e9-4a31-aba3-c268211cc3a9" containerID="1198c48886694faed8f91500a2e8dcf7d577d423ddb6320908fc863111e6656c" exitCode=0
Jul 15 12:08:28.230606 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:28.230589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerDied","Data":"1198c48886694faed8f91500a2e8dcf7d577d423ddb6320908fc863111e6656c"}
Jul 15 12:08:29.237419 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.237378    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh85v" event={"ID":"cae9499a-36e9-4a31-aba3-c268211cc3a9","Type":"ContainerStarted","Data":"b559cfcf91f017d696c4dfcdc4c1be7d9e0a2597a21a7c95f29c5269390f10ff"}
Jul 15 12:08:29.238998 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.238970    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqwzl" event={"ID":"fc6aefad-a945-40b1-8746-e45c0084de59","Type":"ContainerStarted","Data":"1c6e5b936cb6ce2d32b4084eb9c790c427aea85d165582d303f37c8defb60146"}
Jul 15 12:08:29.267240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.267135    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qh85v" podStartSLOduration=7.157998862 podStartE2EDuration="39.267116331s" totalImagesPullingTime="32.109117469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:29.266817909 +0000 UTC m=+40.366767247" watchObservedRunningTime="2026-07-15 12:08:29.267116331 +0000 UTC m=+40.367065668"
Jul 15 12:08:29.284725 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.284671    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hqwzl" podStartSLOduration=1.674875575 podStartE2EDuration="4.284654138s" totalImagesPullingTime="2.609778563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:29.283743385 +0000 UTC m=+40.383692724" watchObservedRunningTime="2026-07-15 12:08:29.284654138 +0000 UTC m=+40.384603501"
Jul 15 12:08:29.325815 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.325770    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:29.325970 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:29.325890    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:08:29.325970 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:29.325953    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls podName:34d1fcc1-504d-4d13-bd09-50803e3981df nodeName:}" failed. No retries permitted until 2026-07-15 12:08:33.325935154 +0000 UTC m=+44.425884488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls") pod "dns-default-7kp2x" (UID: "34d1fcc1-504d-4d13-bd09-50803e3981df") : secret "dns-default-metrics-tls" not found
Jul 15 12:08:29.326070 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.326020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:29.326193 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:29.326174    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:08:29.326250 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:29.326195    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98b64dc76-n5wmb: secret "image-registry-tls" not found
Jul 15 12:08:29.326250 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:29.326235    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls podName:7980fcb6-1d65-4f03-b503-3b2bea12e387 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:33.32622203 +0000 UTC m=+44.426171346 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls") pod "image-registry-98b64dc76-n5wmb" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387") : secret "image-registry-tls" not found
Jul 15 12:08:29.410461 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.410359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:29.492596 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.492558    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:29.596075 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.596026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:29.671563 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:29.671471    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:33.357462 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:33.357422    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:33.357462 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:33.357465    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:33.357988 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:33.357574    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:08:33.357988 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:33.357639    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls podName:34d1fcc1-504d-4d13-bd09-50803e3981df nodeName:}" failed. No retries permitted until 2026-07-15 12:08:41.357622746 +0000 UTC m=+52.457572063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls") pod "dns-default-7kp2x" (UID: "34d1fcc1-504d-4d13-bd09-50803e3981df") : secret "dns-default-metrics-tls" not found
Jul 15 12:08:33.357988 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:33.357581    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:08:33.357988 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:33.357677    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98b64dc76-n5wmb: secret "image-registry-tls" not found
Jul 15 12:08:33.357988 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:33.357722    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls podName:7980fcb6-1d65-4f03-b503-3b2bea12e387 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:41.357709019 +0000 UTC m=+52.457658363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls") pod "image-registry-98b64dc76-n5wmb" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387") : secret "image-registry-tls" not found
Jul 15 12:08:34.312701 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:34.312669    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:34.415032 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:34.414989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:34.760167 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:34.760131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:34.861389 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:34.861352    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:35.755910 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.755872    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-676fc56cdb-2dvx9"]
Jul 15 12:08:35.789133 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.789082    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-676fc56cdb-2dvx9"]
Jul 15 12:08:35.789281 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.789219    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.793109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.793066    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 15 12:08:35.793244 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.793111    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 15 12:08:35.793314 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.793291    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 12:08:35.793419 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.793399    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6lssj\""
Jul 15 12:08:35.793505 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.793490    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 15 12:08:35.793568 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.793557    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 15 12:08:35.793648 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.793625    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 12:08:35.875725 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.875691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.875908 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.875740    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-default-certificate\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.875908 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.875777    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.876020 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.875919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blwpk\" (UniqueName: \"kubernetes.io/projected/a147a563-6d3a-4034-85b2-d05a3b34913c-kube-api-access-blwpk\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.876020 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.875971    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-stats-auth\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.976341 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.976299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.976341 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.976349    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-default-certificate\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.976577 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.976368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.976577 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.976428    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blwpk\" (UniqueName: \"kubernetes.io/projected/a147a563-6d3a-4034-85b2-d05a3b34913c-kube-api-access-blwpk\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.976577 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.976464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-stats-auth\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.976577 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:35.976465    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 12:08:35.976577 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:35.976561    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:36.476543135 +0000 UTC m=+47.576492457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : secret "router-metrics-certs-default" not found
Jul 15 12:08:35.976750 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:35.976586    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:36.476576801 +0000 UTC m=+47.576526135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : configmap references non-existent config key: service-ca.crt
Jul 15 12:08:35.979996 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.979972    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-default-certificate\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.980126 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.980004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-stats-auth\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:35.985748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:35.985717    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blwpk\" (UniqueName: \"kubernetes.io/projected/a147a563-6d3a-4034-85b2-d05a3b34913c-kube-api-access-blwpk\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:36.200329 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:36.200237    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:36.269944 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:36.269905    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:36.480287 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:36.480258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:36.480435 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:36.480302    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:36.480435 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:36.480407    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 12:08:36.480511 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:36.480412    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:37.480399148 +0000 UTC m=+48.580348465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : configmap references non-existent config key: service-ca.crt
Jul 15 12:08:36.480555 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:36.480535    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:37.480520532 +0000 UTC m=+48.580469853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : secret "router-metrics-certs-default" not found
Jul 15 12:08:37.489604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:37.489566    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:37.490006 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:37.489610    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:37.490006 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:37.489735    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:39.489720778 +0000 UTC m=+50.589670095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : configmap references non-existent config key: service-ca.crt
Jul 15 12:08:37.490006 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:37.489741    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 12:08:37.490006 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:37.489818    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:39.489797583 +0000 UTC m=+50.589746914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : secret "router-metrics-certs-default" not found
Jul 15 12:08:38.148745 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.148717    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"]
Jul 15 12:08:38.200882 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.200857    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"]
Jul 15 12:08:38.201018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.200948    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"
Jul 15 12:08:38.207559 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.207536    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6cplc\""
Jul 15 12:08:38.295427 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.295400    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnngv\" (UniqueName: \"kubernetes.io/projected/18bf8905-ca55-4e4f-afb3-c075ce149795-kube-api-access-jnngv\") pod \"network-check-source-869dcfcdcd-th5wb\" (UID: \"18bf8905-ca55-4e4f-afb3-c075ce149795\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"
Jul 15 12:08:38.396110 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.396056    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnngv\" (UniqueName: \"kubernetes.io/projected/18bf8905-ca55-4e4f-afb3-c075ce149795-kube-api-access-jnngv\") pod \"network-check-source-869dcfcdcd-th5wb\" (UID: \"18bf8905-ca55-4e4f-afb3-c075ce149795\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"
Jul 15 12:08:38.409490 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.409417    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnngv\" (UniqueName: \"kubernetes.io/projected/18bf8905-ca55-4e4f-afb3-c075ce149795-kube-api-access-jnngv\") pod \"network-check-source-869dcfcdcd-th5wb\" (UID: \"18bf8905-ca55-4e4f-afb3-c075ce149795\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"
Jul 15 12:08:38.509387 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.509349    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"
Jul 15 12:08:38.653220 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:38.653185    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb"]
Jul 15 12:08:38.656815 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:08:38.656788    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bf8905_ca55_4e4f_afb3_c075ce149795.slice/crio-ab26747b643a65a46ed0290520ab6442291391a0337b4a2b9aca40f0eeac1f36 WatchSource:0}: Error finding container ab26747b643a65a46ed0290520ab6442291391a0337b4a2b9aca40f0eeac1f36: Status 404 returned error can't find the container with id ab26747b643a65a46ed0290520ab6442291391a0337b4a2b9aca40f0eeac1f36
Jul 15 12:08:39.262435 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:39.262399    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb" event={"ID":"18bf8905-ca55-4e4f-afb3-c075ce149795","Type":"ContainerStarted","Data":"ab26747b643a65a46ed0290520ab6442291391a0337b4a2b9aca40f0eeac1f36"}
Jul 15 12:08:39.505718 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:39.505671    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:39.505912 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:39.505850    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 12:08:39.505912 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:39.505866    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:39.506031 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:39.505925    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:43.505903425 +0000 UTC m=+54.605852754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : secret "router-metrics-certs-default" not found
Jul 15 12:08:39.506031 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:39.506009    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:43.505996585 +0000 UTC m=+54.605945905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : configmap references non-existent config key: service-ca.crt
Jul 15 12:08:41.314110 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:41.314064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:41.415264 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:41.415222    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log" limit=1024
Jul 15 12:08:41.421261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:41.421228    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:41.421410 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:41.421351    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:08:41.421410 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:41.421368    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98b64dc76-n5wmb: secret "image-registry-tls" not found
Jul 15 12:08:41.421410 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:41.421379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:41.421548 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:41.421432    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls podName:7980fcb6-1d65-4f03-b503-3b2bea12e387 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:57.421413677 +0000 UTC m=+68.521363015 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls") pod "image-registry-98b64dc76-n5wmb" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387") : secret "image-registry-tls" not found
Jul 15 12:08:41.421548 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:41.421470    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:08:41.421548 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:41.421503    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls podName:34d1fcc1-504d-4d13-bd09-50803e3981df nodeName:}" failed. No retries permitted until 2026-07-15 12:08:57.421493353 +0000 UTC m=+68.521442670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls") pod "dns-default-7kp2x" (UID: "34d1fcc1-504d-4d13-bd09-50803e3981df") : secret "dns-default-metrics-tls" not found
Jul 15 12:08:42.214541 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.214437    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-prhcw"]
Jul 15 12:08:42.234344 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.234299    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.240045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.238357    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-prhcw"]
Jul 15 12:08:42.240045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.238525    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 12:08:42.240045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.238974    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rwqkp\""
Jul 15 12:08:42.240045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.239249    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 12:08:42.240045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.239445    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 12:08:42.240045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.239602    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 12:08:42.272228 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.272190    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb" event={"ID":"18bf8905-ca55-4e4f-afb3-c075ce149795","Type":"ContainerStarted","Data":"2a21a38bab2dd323454854312eff6b809c761331f0e73d04874880f45180030f"}
Jul 15 12:08:42.302042 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.301979    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-th5wb" podStartSLOduration=1.004332557 podStartE2EDuration="4.301955812s" totalImagesPullingTime="3.297623255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:42.301696134 +0000 UTC m=+53.401645483" watchObservedRunningTime="2026-07-15 12:08:42.301955812 +0000 UTC m=+53.401905151"
Jul 15 12:08:42.430784 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.430738    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-data-volume\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.431273 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.430816    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-crio-socket\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.431273 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.430868    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.431273 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.430907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6rn6\" (UniqueName: \"kubernetes.io/projected/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-api-access-h6rn6\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.431273 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.430979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532168 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.532052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-data-volume\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532168 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.532121    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-crio-socket\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532400 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.532265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532400 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.532287    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-crio-socket\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532400 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.532315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6rn6\" (UniqueName: \"kubernetes.io/projected/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-api-access-h6rn6\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532400 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.532353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532400 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:42.532383    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:42.532400 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.532392    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-data-volume\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:42.532666 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:42.532451    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:43.032428736 +0000 UTC m=+54.132378066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:42.532666 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:42.532539    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:08:42.532666 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:42.532599    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:43.03258467 +0000 UTC m=+54.132534001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : configmap "kube-rbac-proxy" not found
Jul 15 12:08:42.543626 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:42.543589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6rn6\" (UniqueName: \"kubernetes.io/projected/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-api-access-h6rn6\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:43.036650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:43.036610    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:43.036827 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:43.036662    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:43.036827 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:43.036733    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:08:43.036827 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:43.036755    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:43.036827 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:43.036784    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:44.036771143 +0000 UTC m=+55.136720459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : configmap "kube-rbac-proxy" not found
Jul 15 12:08:43.036827 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:43.036805    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:44.036792232 +0000 UTC m=+55.136741569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:43.541142 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:43.541081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:43.541497 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:43.541157    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:43.541497 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:43.541227    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 12:08:43.541497 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:43.541292    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:51.541276564 +0000 UTC m=+62.641225881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : secret "router-metrics-certs-default" not found
Jul 15 12:08:43.541497 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:43.541320    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:08:51.541307504 +0000 UTC m=+62.641256820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : configmap references non-existent config key: service-ca.crt
Jul 15 12:08:43.647941 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:43.647914    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7bkzq_728cb636-c45f-42d8-a03b-b5f54a4fd18d/dns-node-resolver/0.log"
Jul 15 12:08:44.045331 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:44.045296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:44.045499 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:44.045342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:44.045499 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:44.045431    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:08:44.045499 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:44.045446    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:44.045499 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:44.045492    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:46.045477105 +0000 UTC m=+57.145426425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : configmap "kube-rbac-proxy" not found
Jul 15 12:08:44.045634 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:44.045506    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:46.045500233 +0000 UTC m=+57.145449549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:44.837401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:44.837373    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sp6l2_69dae2e5-759c-4555-9ac7-b6ced7bb8177/node-ca/0.log"
Jul 15 12:08:45.237860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:45.237831    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqwzl_fc6aefad-a945-40b1-8746-e45c0084de59/serve-healthcheck-canary/0.log"
Jul 15 12:08:46.060055 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:46.060026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:46.060498 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:46.060161    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:08:46.060498 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:46.060172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:46.060498 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:46.060223    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:50.06020557 +0000 UTC m=+61.160154890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : configmap "kube-rbac-proxy" not found
Jul 15 12:08:46.060498 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:46.060263    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:46.060498 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:46.060299    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:50.060290612 +0000 UTC m=+61.160239932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:50.092592 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:50.092540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:50.092968 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:50.092608    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:50.092968 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:50.092687    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:08:50.092968 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:50.092704    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:50.092968 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:50.092770    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:58.092752997 +0000 UTC m=+69.192702326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:50.092968 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:50.092787    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:08:58.092780837 +0000 UTC m=+69.192730199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : configmap "kube-rbac-proxy" not found
Jul 15 12:08:51.215717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:51.215681    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sx9r"
Jul 15 12:08:51.605429 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:51.605343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:51.605429 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:51.605397    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:08:51.605610 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:51.605501    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 12:08:51.605610 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:51.605588    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:09:07.605569969 +0000 UTC m=+78.705519311 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : secret "router-metrics-certs-default" not found
Jul 15 12:08:51.605610 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:51.605603    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle podName:a147a563-6d3a-4034-85b2-d05a3b34913c nodeName:}" failed. No retries permitted until 2026-07-15 12:09:07.605597117 +0000 UTC m=+78.705546433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle") pod "router-default-676fc56cdb-2dvx9" (UID: "a147a563-6d3a-4034-85b2-d05a3b34913c") : configmap references non-existent config key: service-ca.crt
Jul 15 12:08:55.132599 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.132551    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:55.132599 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.132618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:08:55.133119 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.132680    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:55.135103 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.135075    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8txhr\" (UniqueName: \"kubernetes.io/projected/9121b055-df43-4988-8bf1-dd0bb1d5ac08-kube-api-access-8txhr\") pod \"network-check-target-4qbcd\" (UID: \"9121b055-df43-4988-8bf1-dd0bb1d5ac08\") " pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:55.135316 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.135279    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 12:08:55.136595 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.136581    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 12:08:55.143273 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:55.143247    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 12:08:55.143424 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:55.143312    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs podName:8ee8bd80-e264-413e-84d3-1a40c8c412ca nodeName:}" failed. No retries permitted until 2026-07-15 12:09:59.143294256 +0000 UTC m=+130.243243576 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs") pod "network-metrics-daemon-9jcw5" (UID: "8ee8bd80-e264-413e-84d3-1a40c8c412ca") : secret "metrics-daemon-secret" not found
Jul 15 12:08:55.145359 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.145337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac464eb-a63f-470b-883d-93d07e830acb-original-pull-secret\") pod \"global-pull-secret-syncer-h4nz6\" (UID: \"dac464eb-a63f-470b-883d-93d07e830acb\") " pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:55.170667 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.170635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-674f4\""
Jul 15 12:08:55.174579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.174561    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4nz6"
Jul 15 12:08:55.178680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.178665    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:55.307070 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.307042    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4qbcd"]
Jul 15 12:08:55.310717 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:08:55.310690    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9121b055_df43_4988_8bf1_dd0bb1d5ac08.slice/crio-c76a187ac46dae96ef8470fc14e6c8e0b48d40554f61fa9c5bdd2c6e9f864fcf WatchSource:0}: Error finding container c76a187ac46dae96ef8470fc14e6c8e0b48d40554f61fa9c5bdd2c6e9f864fcf: Status 404 returned error can't find the container with id c76a187ac46dae96ef8470fc14e6c8e0b48d40554f61fa9c5bdd2c6e9f864fcf
Jul 15 12:08:55.320925 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:55.320896    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h4nz6"]
Jul 15 12:08:55.324646 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:08:55.324616    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac464eb_a63f_470b_883d_93d07e830acb.slice/crio-ad81517993e266318d988fbe9880451d91f6b0452887cb91e7863c79732f61fa WatchSource:0}: Error finding container ad81517993e266318d988fbe9880451d91f6b0452887cb91e7863c79732f61fa: Status 404 returned error can't find the container with id ad81517993e266318d988fbe9880451d91f6b0452887cb91e7863c79732f61fa
Jul 15 12:08:56.308284 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:56.308245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4qbcd" event={"ID":"9121b055-df43-4988-8bf1-dd0bb1d5ac08","Type":"ContainerStarted","Data":"36653709c1ef0a2588a06e707d3c4cc8f621f9051dd6716306bb6fa45c8fd864"}
Jul 15 12:08:56.308745 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:56.308301    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4qbcd" event={"ID":"9121b055-df43-4988-8bf1-dd0bb1d5ac08","Type":"ContainerStarted","Data":"c76a187ac46dae96ef8470fc14e6c8e0b48d40554f61fa9c5bdd2c6e9f864fcf"}
Jul 15 12:08:56.308745 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:56.308353    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:08:56.309585 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:56.309559    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h4nz6" event={"ID":"dac464eb-a63f-470b-883d-93d07e830acb","Type":"ContainerStarted","Data":"ad81517993e266318d988fbe9880451d91f6b0452887cb91e7863c79732f61fa"}
Jul 15 12:08:56.325546 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:56.325491    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4qbcd" podStartSLOduration=66.325474787 podStartE2EDuration="1m6.325474787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:56.324257148 +0000 UTC m=+67.424206486" watchObservedRunningTime="2026-07-15 12:08:56.325474787 +0000 UTC m=+67.425424128"
Jul 15 12:08:57.452306 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.452248    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:57.452756 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.452314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:57.455220 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.455191    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34d1fcc1-504d-4d13-bd09-50803e3981df-metrics-tls\") pod \"dns-default-7kp2x\" (UID: \"34d1fcc1-504d-4d13-bd09-50803e3981df\") " pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:57.455327 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.455258    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"image-registry-98b64dc76-n5wmb\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") " pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:57.703803 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.703729    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p8szp\""
Jul 15 12:08:57.709922 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.709903    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:57.727017 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.726994    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cbvbt\""
Jul 15 12:08:57.735132 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.735104    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:08:57.866578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.866543    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-98b64dc76-n5wmb"]
Jul 15 12:08:57.870032 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:08:57.870002    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7980fcb6_1d65_4f03_b503_3b2bea12e387.slice/crio-183d8c5168fbdc20b9222521c5e3201765080d4deb017b0e764ffdcac477136f WatchSource:0}: Error finding container 183d8c5168fbdc20b9222521c5e3201765080d4deb017b0e764ffdcac477136f: Status 404 returned error can't find the container with id 183d8c5168fbdc20b9222521c5e3201765080d4deb017b0e764ffdcac477136f
Jul 15 12:08:57.888206 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:57.888181    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7kp2x"]
Jul 15 12:08:57.891412 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:08:57.891385    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d1fcc1_504d_4d13_bd09_50803e3981df.slice/crio-7ee70da679e294f0c3407ecc0f78ea6459d6b4729499ccd85d0f3bf3de577275 WatchSource:0}: Error finding container 7ee70da679e294f0c3407ecc0f78ea6459d6b4729499ccd85d0f3bf3de577275: Status 404 returned error can't find the container with id 7ee70da679e294f0c3407ecc0f78ea6459d6b4729499ccd85d0f3bf3de577275
Jul 15 12:08:58.158932 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.158889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:58.159117 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.158941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:58.159117 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:58.159069    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:58.159217 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:08:58.159172    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls podName:8d0b2eb9-ed8f-4588-87c4-f3184611afa2 nodeName:}" failed. No retries permitted until 2026-07-15 12:09:14.159150115 +0000 UTC m=+85.259099439 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prhcw" (UID: "8d0b2eb9-ed8f-4588-87c4-f3184611afa2") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:08:58.159543 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.159520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:08:58.316851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.316767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb" event={"ID":"7980fcb6-1d65-4f03-b503-3b2bea12e387","Type":"ContainerStarted","Data":"db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9"}
Jul 15 12:08:58.316851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.316836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb" event={"ID":"7980fcb6-1d65-4f03-b503-3b2bea12e387","Type":"ContainerStarted","Data":"183d8c5168fbdc20b9222521c5e3201765080d4deb017b0e764ffdcac477136f"}
Jul 15 12:08:58.317336 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.317308    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:08:58.318581 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.318553    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kp2x" event={"ID":"34d1fcc1-504d-4d13-bd09-50803e3981df","Type":"ContainerStarted","Data":"7ee70da679e294f0c3407ecc0f78ea6459d6b4729499ccd85d0f3bf3de577275"}
Jul 15 12:08:58.345542 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:08:58.345483    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb" podStartSLOduration=67.345463984 podStartE2EDuration="1m7.345463984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:58.344445955 +0000 UTC m=+69.444395294" watchObservedRunningTime="2026-07-15 12:08:58.345463984 +0000 UTC m=+69.445413321"
Jul 15 12:09:00.327353 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:00.327310    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h4nz6" event={"ID":"dac464eb-a63f-470b-883d-93d07e830acb","Type":"ContainerStarted","Data":"6f77b482ec44f810e73de6eebcde9d21881753289d927f91999bba1ad13510ce"}
Jul 15 12:09:00.351593 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:00.351533    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h4nz6" podStartSLOduration=65.533584679 podStartE2EDuration="1m10.35151199s" totalImagesPullingTime="4.817927311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:00.351057016 +0000 UTC m=+71.451006360" watchObservedRunningTime="2026-07-15 12:09:00.35151199 +0000 UTC m=+71.451461326"
Jul 15 12:09:02.335471 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:02.335440    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kp2x" event={"ID":"34d1fcc1-504d-4d13-bd09-50803e3981df","Type":"ContainerStarted","Data":"c21cd711b467804c4f7fdf4649ec36c67604f55dd415c560a8b2ba198f6a2485"}
Jul 15 12:09:02.335887 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:02.335491    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kp2x" event={"ID":"34d1fcc1-504d-4d13-bd09-50803e3981df","Type":"ContainerStarted","Data":"8de59f2ce585f9158afd35c1c9d356dde227740d899e6150e17dbc3e1cd9a7bf"}
Jul 15 12:09:02.335887 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:02.335601    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:09:02.355591 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:02.355544    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7kp2x" podStartSLOduration=33.590838021 podStartE2EDuration="37.355525907s" totalImagesPullingTime="3.764687886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:02.354887654 +0000 UTC m=+73.454836992" watchObservedRunningTime="2026-07-15 12:09:02.355525907 +0000 UTC m=+73.455475248"
Jul 15 12:09:07.630907 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:07.630871    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:07.631319 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:07.630952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:07.631533 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:07.631512    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a147a563-6d3a-4034-85b2-d05a3b34913c-service-ca-bundle\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:07.633140 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:07.633125    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a147a563-6d3a-4034-85b2-d05a3b34913c-metrics-certs\") pod \"router-default-676fc56cdb-2dvx9\" (UID: \"a147a563-6d3a-4034-85b2-d05a3b34913c\") " pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:07.901266 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:07.901186    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6lssj\""
Jul 15 12:09:07.909284 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:07.909262    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:08.037724 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:08.037693    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-676fc56cdb-2dvx9"]
Jul 15 12:09:08.041032 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:08.041007    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda147a563_6d3a_4034_85b2_d05a3b34913c.slice/crio-b72e9078a838c2f2a90f3d1ffae9886448cab71fe3210f0d2fa408848dac81d1 WatchSource:0}: Error finding container b72e9078a838c2f2a90f3d1ffae9886448cab71fe3210f0d2fa408848dac81d1: Status 404 returned error can't find the container with id b72e9078a838c2f2a90f3d1ffae9886448cab71fe3210f0d2fa408848dac81d1
Jul 15 12:09:08.354381 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:08.354346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-676fc56cdb-2dvx9" event={"ID":"a147a563-6d3a-4034-85b2-d05a3b34913c","Type":"ContainerStarted","Data":"69e2f13e744815ed478df44b44a7c9378d92a19581cd3db955fa5e03f5e71b49"}
Jul 15 12:09:08.354577 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:08.354395    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-676fc56cdb-2dvx9" event={"ID":"a147a563-6d3a-4034-85b2-d05a3b34913c","Type":"ContainerStarted","Data":"b72e9078a838c2f2a90f3d1ffae9886448cab71fe3210f0d2fa408848dac81d1"}
Jul 15 12:09:08.909807 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:08.909765    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:08.912423 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:08.912397    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:08.939888 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:08.939828    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-676fc56cdb-2dvx9" podStartSLOduration=33.939809974 podStartE2EDuration="33.939809974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:08.380944247 +0000 UTC m=+79.480893609" watchObservedRunningTime="2026-07-15 12:09:08.939809974 +0000 UTC m=+80.039759318"
Jul 15 12:09:09.359903 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.359865    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:09.362037 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.362012    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-676fc56cdb-2dvx9"
Jul 15 12:09:09.729297 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.729268    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-j648n"]
Jul 15 12:09:09.732112 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.732061    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:09.739011 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.738990    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 15 12:09:09.740612 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.740592    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"]
Jul 15 12:09:09.743418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.743401    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:09.747782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.747759    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 15 12:09:09.756299 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:09.756264    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-dockercfg-q685s\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-q685s\"" type="*v1.Secret"
Jul 15 12:09:09.756469 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:09.756354    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 15 12:09:09.756609 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:09.756403    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret"
Jul 15 12:09:09.757193 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:09.756455    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 15 12:09:09.757193 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.757135    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6l6fh\""
Jul 15 12:09:09.758045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.758025    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-j648n"]
Jul 15 12:09:09.758834 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.758813    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"]
Jul 15 12:09:09.760683 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.760664    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 15 12:09:09.762051 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.762032    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.764387 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.764366    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 15 12:09:09.777120 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.777065    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"]
Jul 15 12:09:09.777925 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.777905    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"]
Jul 15 12:09:09.849938 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.849897    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6fddb889-36c7-48a3-bb1f-d5f5245382ff-klusterlet-config\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.849938 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.849938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0dfbf433-bb2f-44d7-b404-f481b75adaa4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9886c7994-5tzlf\" (UID: \"0dfbf433-bb2f-44d7-b404-f481b75adaa4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:09.850224 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.849958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5af6bb98-fdb6-44aa-b77a-ec0027e2b90a-nginx-conf\") pod \"networking-console-plugin-6cf856758c-j648n\" (UID: \"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:09.850224 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.850060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5af6bb98-fdb6-44aa-b77a-ec0027e2b90a-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-j648n\" (UID: \"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:09.850224 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.850148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frgjj\" (UniqueName: \"kubernetes.io/projected/0dfbf433-bb2f-44d7-b404-f481b75adaa4-kube-api-access-frgjj\") pod \"managed-serviceaccount-addon-agent-9886c7994-5tzlf\" (UID: \"0dfbf433-bb2f-44d7-b404-f481b75adaa4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:09.850224 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.850198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fddb889-36c7-48a3-bb1f-d5f5245382ff-tmp\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.850371 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.850222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7wn\" (UniqueName: \"kubernetes.io/projected/6fddb889-36c7-48a3-bb1f-d5f5245382ff-kube-api-access-6v7wn\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.875958 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.875922    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-c95ck"]
Jul 15 12:09:09.878917 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.878894    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-c95ck"
Jul 15 12:09:09.881672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.881653    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 12:09:09.882322 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.882306    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-dl4x6\""
Jul 15 12:09:09.882419 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.882303    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 12:09:09.893768 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.893743    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-c95ck"]
Jul 15 12:09:09.951284 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6fddb889-36c7-48a3-bb1f-d5f5245382ff-klusterlet-config\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.951284 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0dfbf433-bb2f-44d7-b404-f481b75adaa4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9886c7994-5tzlf\" (UID: \"0dfbf433-bb2f-44d7-b404-f481b75adaa4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:09.951716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5af6bb98-fdb6-44aa-b77a-ec0027e2b90a-nginx-conf\") pod \"networking-console-plugin-6cf856758c-j648n\" (UID: \"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:09.951716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951566    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5af6bb98-fdb6-44aa-b77a-ec0027e2b90a-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-j648n\" (UID: \"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:09.951716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951612    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st8d5\" (UniqueName: \"kubernetes.io/projected/05775dd6-2844-48cb-a259-99f67edf039a-kube-api-access-st8d5\") pod \"downloads-68c9cbb497-c95ck\" (UID: \"05775dd6-2844-48cb-a259-99f67edf039a\") " pod="openshift-console/downloads-68c9cbb497-c95ck"
Jul 15 12:09:09.951716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frgjj\" (UniqueName: \"kubernetes.io/projected/0dfbf433-bb2f-44d7-b404-f481b75adaa4-kube-api-access-frgjj\") pod \"managed-serviceaccount-addon-agent-9886c7994-5tzlf\" (UID: \"0dfbf433-bb2f-44d7-b404-f481b75adaa4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:09.951716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fddb889-36c7-48a3-bb1f-d5f5245382ff-tmp\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.951716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.951701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7wn\" (UniqueName: \"kubernetes.io/projected/6fddb889-36c7-48a3-bb1f-d5f5245382ff-kube-api-access-6v7wn\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.952108 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.952068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5af6bb98-fdb6-44aa-b77a-ec0027e2b90a-nginx-conf\") pod \"networking-console-plugin-6cf856758c-j648n\" (UID: \"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:09.952223 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.952201    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fddb889-36c7-48a3-bb1f-d5f5245382ff-tmp\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.953976 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.953958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6fddb889-36c7-48a3-bb1f-d5f5245382ff-klusterlet-config\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:09.954293 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.954273    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5af6bb98-fdb6-44aa-b77a-ec0027e2b90a-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-j648n\" (UID: \"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:09.954370 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:09.954354    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0dfbf433-bb2f-44d7-b404-f481b75adaa4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9886c7994-5tzlf\" (UID: \"0dfbf433-bb2f-44d7-b404-f481b75adaa4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:10.041345 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.041248    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n"
Jul 15 12:09:10.052019 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.051994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st8d5\" (UniqueName: \"kubernetes.io/projected/05775dd6-2844-48cb-a259-99f67edf039a-kube-api-access-st8d5\") pod \"downloads-68c9cbb497-c95ck\" (UID: \"05775dd6-2844-48cb-a259-99f67edf039a\") " pod="openshift-console/downloads-68c9cbb497-c95ck"
Jul 15 12:09:10.065702 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.065668    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st8d5\" (UniqueName: \"kubernetes.io/projected/05775dd6-2844-48cb-a259-99f67edf039a-kube-api-access-st8d5\") pod \"downloads-68c9cbb497-c95ck\" (UID: \"05775dd6-2844-48cb-a259-99f67edf039a\") " pod="openshift-console/downloads-68c9cbb497-c95ck"
Jul 15 12:09:10.172509 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.172484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-j648n"]
Jul 15 12:09:10.174523 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:10.174501    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af6bb98_fdb6_44aa_b77a_ec0027e2b90a.slice/crio-0bf9bc450094a64b5724c68056f054a4a3c3499d150133749ebde5b48f089a8f WatchSource:0}: Error finding container 0bf9bc450094a64b5724c68056f054a4a3c3499d150133749ebde5b48f089a8f: Status 404 returned error can't find the container with id 0bf9bc450094a64b5724c68056f054a4a3c3499d150133749ebde5b48f089a8f
Jul 15 12:09:10.187564 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.187532    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-c95ck"
Jul 15 12:09:10.321991 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.321822    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-c95ck"]
Jul 15 12:09:10.326420 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:10.326392    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05775dd6_2844_48cb_a259_99f67edf039a.slice/crio-27d446fa861a65ad861fa51220ef71116dfb87596c780cb5dfff43f9c7a271c3 WatchSource:0}: Error finding container 27d446fa861a65ad861fa51220ef71116dfb87596c780cb5dfff43f9c7a271c3: Status 404 returned error can't find the container with id 27d446fa861a65ad861fa51220ef71116dfb87596c780cb5dfff43f9c7a271c3
Jul 15 12:09:10.362246 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.362210    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n" event={"ID":"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a","Type":"ContainerStarted","Data":"0bf9bc450094a64b5724c68056f054a4a3c3499d150133749ebde5b48f089a8f"}
Jul 15 12:09:10.363298 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.363274    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-c95ck" event={"ID":"05775dd6-2844-48cb-a259-99f67edf039a","Type":"ContainerStarted","Data":"27d446fa861a65ad861fa51220ef71116dfb87596c780cb5dfff43f9c7a271c3"}
Jul 15 12:09:10.617358 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.617274    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 12:09:10.635942 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.635909    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"]
Jul 15 12:09:10.639840 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.639824    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"
Jul 15 12:09:10.642629 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.642609    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 12:09:10.642830 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.642812    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cfsp6\""
Jul 15 12:09:10.653509 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.653487    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"]
Jul 15 12:09:10.757064 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.757027    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/75996646-d932-4bca-ba39-06bf99ac8a46-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-dr7tr\" (UID: \"75996646-d932-4bca-ba39-06bf99ac8a46\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"
Jul 15 12:09:10.858049 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.858012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/75996646-d932-4bca-ba39-06bf99ac8a46-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-dr7tr\" (UID: \"75996646-d932-4bca-ba39-06bf99ac8a46\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"
Jul 15 12:09:10.860857 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.860824    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/75996646-d932-4bca-ba39-06bf99ac8a46-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-dr7tr\" (UID: \"75996646-d932-4bca-ba39-06bf99ac8a46\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"
Jul 15 12:09:10.918401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.918333    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 12:09:10.948707 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:10.948674    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"
Jul 15 12:09:10.961628 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:10.961116    2582 projected.go:291] Couldn't get configMap open-cluster-management-agent-addon/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 15 12:09:10.961628 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:10.961159    2582 projected.go:196] Error preparing data for projected volume kube-api-access-frgjj for pod open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf: failed to sync configmap cache: timed out waiting for the condition
Jul 15 12:09:10.961628 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:10.961224    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dfbf433-bb2f-44d7-b404-f481b75adaa4-kube-api-access-frgjj podName:0dfbf433-bb2f-44d7-b404-f481b75adaa4 nodeName:}" failed. No retries permitted until 2026-07-15 12:09:11.461201563 +0000 UTC m=+82.561150898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-frgjj" (UniqueName: "kubernetes.io/projected/0dfbf433-bb2f-44d7-b404-f481b75adaa4-kube-api-access-frgjj") pod "managed-serviceaccount-addon-agent-9886c7994-5tzlf" (UID: "0dfbf433-bb2f-44d7-b404-f481b75adaa4") : failed to sync configmap cache: timed out waiting for the condition
Jul 15 12:09:10.965412 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:10.965386    2582 projected.go:291] Couldn't get configMap open-cluster-management-agent-addon/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 15 12:09:10.965546 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:10.965425    2582 projected.go:196] Error preparing data for projected volume kube-api-access-6v7wn for pod open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m: failed to sync configmap cache: timed out waiting for the condition
Jul 15 12:09:10.965546 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:10.965498    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fddb889-36c7-48a3-bb1f-d5f5245382ff-kube-api-access-6v7wn podName:6fddb889-36c7-48a3-bb1f-d5f5245382ff nodeName:}" failed. No retries permitted until 2026-07-15 12:09:11.465479728 +0000 UTC m=+82.565429055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6v7wn" (UniqueName: "kubernetes.io/projected/6fddb889-36c7-48a3-bb1f-d5f5245382ff-kube-api-access-6v7wn") pod "klusterlet-addon-workmgr-6978c6f54-k4b5m" (UID: "6fddb889-36c7-48a3-bb1f-d5f5245382ff") : failed to sync configmap cache: timed out waiting for the condition
Jul 15 12:09:11.115978 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.115947    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"]
Jul 15 12:09:11.119719 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:11.119691    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75996646_d932_4bca_ba39_06bf99ac8a46.slice/crio-f218f7931e4e229c2bf08519d55ea0ad7baef4dd412e2e601e8d1ed627f32901 WatchSource:0}: Error finding container f218f7931e4e229c2bf08519d55ea0ad7baef4dd412e2e601e8d1ed627f32901: Status 404 returned error can't find the container with id f218f7931e4e229c2bf08519d55ea0ad7baef4dd412e2e601e8d1ed627f32901
Jul 15 12:09:11.229910 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.229870    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-q685s\""
Jul 15 12:09:11.358182 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.358148    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 12:09:11.368310 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.368271    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr" event={"ID":"75996646-d932-4bca-ba39-06bf99ac8a46","Type":"ContainerStarted","Data":"f218f7931e4e229c2bf08519d55ea0ad7baef4dd412e2e601e8d1ed627f32901"}
Jul 15 12:09:11.465221 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.465113    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frgjj\" (UniqueName: \"kubernetes.io/projected/0dfbf433-bb2f-44d7-b404-f481b75adaa4-kube-api-access-frgjj\") pod \"managed-serviceaccount-addon-agent-9886c7994-5tzlf\" (UID: \"0dfbf433-bb2f-44d7-b404-f481b75adaa4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:11.467896 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.467866    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frgjj\" (UniqueName: \"kubernetes.io/projected/0dfbf433-bb2f-44d7-b404-f481b75adaa4-kube-api-access-frgjj\") pod \"managed-serviceaccount-addon-agent-9886c7994-5tzlf\" (UID: \"0dfbf433-bb2f-44d7-b404-f481b75adaa4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:11.562303 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.562213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"
Jul 15 12:09:11.566657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.566584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7wn\" (UniqueName: \"kubernetes.io/projected/6fddb889-36c7-48a3-bb1f-d5f5245382ff-kube-api-access-6v7wn\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:11.569590 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.569560    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7wn\" (UniqueName: \"kubernetes.io/projected/6fddb889-36c7-48a3-bb1f-d5f5245382ff-kube-api-access-6v7wn\") pod \"klusterlet-addon-workmgr-6978c6f54-k4b5m\" (UID: \"6fddb889-36c7-48a3-bb1f-d5f5245382ff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:11.571660 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.571582    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:11.842874 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.842800    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"]
Jul 15 12:09:11.846632 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:11.846606    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fddb889_36c7_48a3_bb1f_d5f5245382ff.slice/crio-b11e1d2b41dced97b5cbf4573cb2a45c967da91fdb9176c31416b4e5f546442b WatchSource:0}: Error finding container b11e1d2b41dced97b5cbf4573cb2a45c967da91fdb9176c31416b4e5f546442b: Status 404 returned error can't find the container with id b11e1d2b41dced97b5cbf4573cb2a45c967da91fdb9176c31416b4e5f546442b
Jul 15 12:09:11.855689 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:11.855665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf"]
Jul 15 12:09:11.859628 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:11.859594    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0dfbf433_bb2f_44d7_b404_f481b75adaa4.slice/crio-c3ab9f222af99428d494b237faaa997c74a25c69e5ced44b303e587fc0167249 WatchSource:0}: Error finding container c3ab9f222af99428d494b237faaa997c74a25c69e5ced44b303e587fc0167249: Status 404 returned error can't find the container with id c3ab9f222af99428d494b237faaa997c74a25c69e5ced44b303e587fc0167249
Jul 15 12:09:12.342768 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:12.341669    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7kp2x"
Jul 15 12:09:12.374544 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:12.374431    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf" event={"ID":"0dfbf433-bb2f-44d7-b404-f481b75adaa4","Type":"ContainerStarted","Data":"c3ab9f222af99428d494b237faaa997c74a25c69e5ced44b303e587fc0167249"}
Jul 15 12:09:12.377578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:12.377358    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n" event={"ID":"5af6bb98-fdb6-44aa-b77a-ec0027e2b90a","Type":"ContainerStarted","Data":"98781ced72c2ddb913f44b68549fd6c5c774d0f99ee7bbdbca63ed7cf44c2cbe"}
Jul 15 12:09:12.381051 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:12.381028    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m" event={"ID":"6fddb889-36c7-48a3-bb1f-d5f5245382ff","Type":"ContainerStarted","Data":"b11e1d2b41dced97b5cbf4573cb2a45c967da91fdb9176c31416b4e5f546442b"}
Jul 15 12:09:13.391314 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:13.391280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr" event={"ID":"75996646-d932-4bca-ba39-06bf99ac8a46","Type":"ContainerStarted","Data":"cbc9dc2963b6aa8091b94b255506f44f1a23a080c4939cf1b1270352f2b9a3de"}
Jul 15 12:09:13.391763 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:13.391635    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"
Jul 15 12:09:13.398278 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:13.398224    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr"
Jul 15 12:09:13.409982 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:13.409294    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-j648n" podStartSLOduration=2.498786941 podStartE2EDuration="4.409273121s" totalImagesPullingTime="1.91048618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:12.399966223 +0000 UTC m=+83.499915563" watchObservedRunningTime="2026-07-15 12:09:13.409273121 +0000 UTC m=+84.509222465"
Jul 15 12:09:13.409982 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:13.409423    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-dr7tr" podStartSLOduration=1.711795653 podStartE2EDuration="3.409410673s" totalImagesPullingTime="1.69761502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:13.407832978 +0000 UTC m=+84.507782319" watchObservedRunningTime="2026-07-15 12:09:13.409410673 +0000 UTC m=+84.509359991"
Jul 15 12:09:14.191868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:14.191717    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:09:14.194673 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:14.194645    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d0b2eb9-ed8f-4588-87c4-f3184611afa2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prhcw\" (UID: \"8d0b2eb9-ed8f-4588-87c4-f3184611afa2\") " pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:09:14.351395 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:14.351362    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rwqkp\""
Jul 15 12:09:14.359823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:14.359790    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-prhcw"
Jul 15 12:09:16.395968 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.395933    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-prhcw"]
Jul 15 12:09:16.399743 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:16.399713    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0b2eb9_ed8f_4588_87c4_f3184611afa2.slice/crio-9cd1c9ce61ebb811c9d68f5bfc8959e45d34f1b2e1349e39a8fd5aaba458ddbd WatchSource:0}: Error finding container 9cd1c9ce61ebb811c9d68f5bfc8959e45d34f1b2e1349e39a8fd5aaba458ddbd: Status 404 returned error can't find the container with id 9cd1c9ce61ebb811c9d68f5bfc8959e45d34f1b2e1349e39a8fd5aaba458ddbd
Jul 15 12:09:16.408644 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.408084    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m" event={"ID":"6fddb889-36c7-48a3-bb1f-d5f5245382ff","Type":"ContainerStarted","Data":"04d49e03fcef9d7d2d83c9f8eac1d7a2dacfbf83e81e90dd3b67c5f03d440d79"}
Jul 15 12:09:16.408644 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.408604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:16.410008 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.409989    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m"
Jul 15 12:09:16.410512 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.410489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prhcw" event={"ID":"8d0b2eb9-ed8f-4588-87c4-f3184611afa2","Type":"ContainerStarted","Data":"9cd1c9ce61ebb811c9d68f5bfc8959e45d34f1b2e1349e39a8fd5aaba458ddbd"}
Jul 15 12:09:16.412518 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.412465    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf" event={"ID":"0dfbf433-bb2f-44d7-b404-f481b75adaa4","Type":"ContainerStarted","Data":"2fa377c4d556db66c711df588ce279ad1d3609c3954141fe1d3539b051e10a0b"}
Jul 15 12:09:16.425352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.425290    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6978c6f54-k4b5m" podStartSLOduration=2.992406654 podStartE2EDuration="7.425272756s" totalImagesPullingTime="4.432866102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:16.424772697 +0000 UTC m=+87.524722037" watchObservedRunningTime="2026-07-15 12:09:16.425272756 +0000 UTC m=+87.525222097"
Jul 15 12:09:16.459880 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:16.459824    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9886c7994-5tzlf" podStartSLOduration=3.056833991 podStartE2EDuration="7.459810358s" totalImagesPullingTime="4.402976367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:16.45850556 +0000 UTC m=+87.558454913" watchObservedRunningTime="2026-07-15 12:09:16.459810358 +0000 UTC m=+87.559759696"
Jul 15 12:09:17.716587 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:17.716543    2582 patch_prober.go:28] interesting pod/image-registry-98b64dc76-n5wmb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 15 12:09:17.717040 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:17.716618    2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb" podUID="7980fcb6-1d65-4f03-b503-3b2bea12e387" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 12:09:19.327580 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:19.327549    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:09:21.581360 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.581326    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mg5n5"]
Jul 15 12:09:21.585695 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.585193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.589309 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.589114    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 12:09:21.589309 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.589188    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 12:09:21.589309 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.589207    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 12:09:21.590021 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.589867    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pv4lp\""
Jul 15 12:09:21.590021 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.589868    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 12:09:21.590218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.590154    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 12:09:21.591272 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.591253    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 12:09:21.657871 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.657834    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-accelerators-collector-config\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658056 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.657913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-wtmp\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658056 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.657942    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7748a1e0-41c6-46d7-9949-1ebd6c67f073-metrics-client-ca\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658056 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.657996    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-tls\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.658058    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-textfile\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.658110    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz27t\" (UniqueName: \"kubernetes.io/projected/7748a1e0-41c6-46d7-9949-1ebd6c67f073-kube-api-access-hz27t\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.658157    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.658203    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-root\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.658261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.658239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-sys\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759363 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759324    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-root\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759530 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759382    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-sys\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759530 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759418    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-accelerators-collector-config\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759530 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-root\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759530 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-wtmp\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759530 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7748a1e0-41c6-46d7-9949-1ebd6c67f073-metrics-client-ca\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-tls\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-sys\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759592    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-textfile\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759609    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-wtmp\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759621    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz27t\" (UniqueName: \"kubernetes.io/projected/7748a1e0-41c6-46d7-9949-1ebd6c67f073-kube-api-access-hz27t\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.759792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.759672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.760076 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:21.760020    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 12:09:21.760150 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:21.760089    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-tls podName:7748a1e0-41c6-46d7-9949-1ebd6c67f073 nodeName:}" failed. No retries permitted until 2026-07-15 12:09:22.260065904 +0000 UTC m=+93.360015235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-tls") pod "node-exporter-mg5n5" (UID: "7748a1e0-41c6-46d7-9949-1ebd6c67f073") : secret "node-exporter-tls" not found
Jul 15 12:09:21.760206 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.760179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7748a1e0-41c6-46d7-9949-1ebd6c67f073-metrics-client-ca\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.760470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.760419    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-accelerators-collector-config\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.760594 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.760518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-textfile\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.762806 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.762783    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:21.802703 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:21.802679    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz27t\" (UniqueName: \"kubernetes.io/projected/7748a1e0-41c6-46d7-9949-1ebd6c67f073-kube-api-access-hz27t\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:22.262903 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:22.262869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-tls\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:22.265445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:22.265415    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7748a1e0-41c6-46d7-9949-1ebd6c67f073-node-exporter-tls\") pod \"node-exporter-mg5n5\" (UID: \"7748a1e0-41c6-46d7-9949-1ebd6c67f073\") " pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:22.438902 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:22.438801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prhcw" event={"ID":"8d0b2eb9-ed8f-4588-87c4-f3184611afa2","Type":"ContainerStarted","Data":"66e7b476a68f7baeb28b920d629fa2ded45a0de779e4618da7a998cf21dc32ac"}
Jul 15 12:09:22.498160 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:22.498123    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mg5n5"
Jul 15 12:09:22.512270 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:22.512226    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7748a1e0_41c6_46d7_9949_1ebd6c67f073.slice/crio-3f1947b0db8fddc67c6f599d627d9f6cd4dc641f63fc9dd803b80fc0f7c2d785 WatchSource:0}: Error finding container 3f1947b0db8fddc67c6f599d627d9f6cd4dc641f63fc9dd803b80fc0f7c2d785: Status 404 returned error can't find the container with id 3f1947b0db8fddc67c6f599d627d9f6cd4dc641f63fc9dd803b80fc0f7c2d785
Jul 15 12:09:23.445375 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:23.445331    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mg5n5" event={"ID":"7748a1e0-41c6-46d7-9949-1ebd6c67f073","Type":"ContainerStarted","Data":"3f1947b0db8fddc67c6f599d627d9f6cd4dc641f63fc9dd803b80fc0f7c2d785"}
Jul 15 12:09:27.314837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:27.314808    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4qbcd"
Jul 15 12:09:33.151004 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:33.150970    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-98b64dc76-n5wmb"]
Jul 15 12:09:33.483344 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:33.483309    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-c95ck" event={"ID":"05775dd6-2844-48cb-a259-99f67edf039a","Type":"ContainerStarted","Data":"a3d7f01b7d882eee22c016245d379546b28ae01a79009e13d7a18fb9fc3d9fb5"}
Jul 15 12:09:33.483528 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:33.483493    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-c95ck"
Jul 15 12:09:33.484945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:33.484917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mg5n5" event={"ID":"7748a1e0-41c6-46d7-9949-1ebd6c67f073","Type":"ContainerStarted","Data":"078b2c8cf6f97b2b42acd008709f37042496d2f4fe17b9e22e96ca7fc0445d99"}
Jul 15 12:09:33.485184 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:33.485155    2582 patch_prober.go:28] interesting pod/downloads-68c9cbb497-c95ck container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" start-of-body=
Jul 15 12:09:33.485257 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:33.485198    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-c95ck" podUID="05775dd6-2844-48cb-a259-99f67edf039a" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused"
Jul 15 12:09:33.503082 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:33.503034    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-c95ck" podStartSLOduration=1.562790273 podStartE2EDuration="24.503015628s" totalImagesPullingTime="22.940225355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:33.501778574 +0000 UTC m=+104.601727912" watchObservedRunningTime="2026-07-15 12:09:33.503015628 +0000 UTC m=+104.602964967"
Jul 15 12:09:34.493610 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:34.493566    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prhcw" event={"ID":"8d0b2eb9-ed8f-4588-87c4-f3184611afa2","Type":"ContainerStarted","Data":"04570e2a652f32c8086a9b60d83e7dd83d798203b2eacbfd3de836412976b1e5"}
Jul 15 12:09:34.496361 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:34.495927    2582 generic.go:356] "Generic (PLEG): container finished" podID="7748a1e0-41c6-46d7-9949-1ebd6c67f073" containerID="078b2c8cf6f97b2b42acd008709f37042496d2f4fe17b9e22e96ca7fc0445d99" exitCode=0
Jul 15 12:09:34.496513 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:34.496363    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mg5n5" event={"ID":"7748a1e0-41c6-46d7-9949-1ebd6c67f073","Type":"ContainerDied","Data":"078b2c8cf6f97b2b42acd008709f37042496d2f4fe17b9e22e96ca7fc0445d99"}
Jul 15 12:09:34.509680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:34.509616    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-c95ck"
Jul 15 12:09:35.503857 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:35.503763    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mg5n5" event={"ID":"7748a1e0-41c6-46d7-9949-1ebd6c67f073","Type":"ContainerStarted","Data":"55e1dc67e72c7aa6d028a88dc14f9e3ab2df5b740c77fdd4713209b1ba39f86f"}
Jul 15 12:09:35.503857 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:35.503837    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mg5n5" event={"ID":"7748a1e0-41c6-46d7-9949-1ebd6c67f073","Type":"ContainerStarted","Data":"ee78d8da835adaf2bb10580d1d3a2bda7418ff4e471b8b8b3879d978534bd60d"}
Jul 15 12:09:35.528015 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:35.527585    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mg5n5" podStartSLOduration=3.669008437 podStartE2EDuration="14.527563582s" totalImagesPullingTime="10.753731674s" totalInitContainerRuntime="104.823471ms" isStatefulPod=true podCreationTimestamp="2026-07-15 12:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:35.52539172 +0000 UTC m=+106.625341059" watchObservedRunningTime="2026-07-15 12:09:35.527563582 +0000 UTC m=+106.627512920"
Jul 15 12:09:39.522328 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:39.522289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prhcw" event={"ID":"8d0b2eb9-ed8f-4588-87c4-f3184611afa2","Type":"ContainerStarted","Data":"6ec4a8fdb50850cbd11783f38f0bdaa4116f36a874f8022bec5e88acda57473c"}
Jul 15 12:09:39.547261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:39.547209    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-prhcw" podStartSLOduration=35.576086039 podStartE2EDuration="57.547194901s" totalImagesPullingTime="21.971108862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:08:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:09:39.543276577 +0000 UTC m=+110.643225914" watchObservedRunningTime="2026-07-15 12:09:39.547194901 +0000 UTC m=+110.647144238"
Jul 15 12:09:58.174821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.174780    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb" podUID="7980fcb6-1d65-4f03-b503-3b2bea12e387" containerName="registry" containerID="cri-o://db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9" gracePeriod=30
Jul 15 12:09:58.455296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.455267    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:09:58.569872 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.569825    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-certificates\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.569872 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.569867    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-image-registry-private-configuration\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.570142 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.569893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jm5n\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-kube-api-access-2jm5n\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.570142 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.569913    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-bound-sa-token\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.570142 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.569970    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.570142 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.570123    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7980fcb6-1d65-4f03-b503-3b2bea12e387-ca-trust-extracted\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.570343 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.570164    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-installation-pull-secrets\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.570343 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.570197    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-trusted-ca\") pod \"7980fcb6-1d65-4f03-b503-3b2bea12e387\" (UID: \"7980fcb6-1d65-4f03-b503-3b2bea12e387\") "
Jul 15 12:09:58.570484 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.570456    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:09:58.570778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.570738    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:09:58.572933 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.572869    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:09:58.573046 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.572953    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:09:58.573046 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.572962    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-kube-api-access-2jm5n" (OuterVolumeSpecName: "kube-api-access-2jm5n") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "kube-api-access-2jm5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:09:58.573046 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.573000    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:09:58.573229 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.573060    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:09:58.578397 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.578371    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7980fcb6-1d65-4f03-b503-3b2bea12e387-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7980fcb6-1d65-4f03-b503-3b2bea12e387" (UID: "7980fcb6-1d65-4f03-b503-3b2bea12e387"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:09:58.591885 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.591859    2582 generic.go:356] "Generic (PLEG): container finished" podID="7980fcb6-1d65-4f03-b503-3b2bea12e387" containerID="db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9" exitCode=0
Jul 15 12:09:58.591989 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.591901    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb"
Jul 15 12:09:58.591989 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.591933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb" event={"ID":"7980fcb6-1d65-4f03-b503-3b2bea12e387","Type":"ContainerDied","Data":"db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9"}
Jul 15 12:09:58.591989 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.591973    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98b64dc76-n5wmb" event={"ID":"7980fcb6-1d65-4f03-b503-3b2bea12e387","Type":"ContainerDied","Data":"183d8c5168fbdc20b9222521c5e3201765080d4deb017b0e764ffdcac477136f"}
Jul 15 12:09:58.592082 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.591993    2582 scope.go:117] "RemoveContainer" containerID="db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9"
Jul 15 12:09:58.600442 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.600423    2582 scope.go:117] "RemoveContainer" containerID="db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9"
Jul 15 12:09:58.600684 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:09:58.600666    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9\": container with ID starting with db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9 not found: ID does not exist" containerID="db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9"
Jul 15 12:09:58.600727 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.600693    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9"} err="failed to get container status \"db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9\": rpc error: code = NotFound desc = could not find container \"db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9\": container with ID starting with db49faeaaae4dca2dd09f48df455f68aaa2bbda7eb4bffb99180e32f7f0f1bc9 not found: ID does not exist"
Jul 15 12:09:58.611566 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.611544    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-98b64dc76-n5wmb"]
Jul 15 12:09:58.615875 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.615855    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-98b64dc76-n5wmb"]
Jul 15 12:09:58.671394 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671363    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-certificates\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:58.671394 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671390    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-image-registry-private-configuration\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:58.671394 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671400    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jm5n\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-kube-api-access-2jm5n\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:58.671579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671409    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-bound-sa-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:58.671579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671417    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7980fcb6-1d65-4f03-b503-3b2bea12e387-registry-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:58.671579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671425    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7980fcb6-1d65-4f03-b503-3b2bea12e387-ca-trust-extracted\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:58.671579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671435    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7980fcb6-1d65-4f03-b503-3b2bea12e387-installation-pull-secrets\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:58.671579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:58.671443    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7980fcb6-1d65-4f03-b503-3b2bea12e387-trusted-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:09:59.175775 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:59.175734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:09:59.177919 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:59.177899    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee8bd80-e264-413e-84d3-1a40c8c412ca-metrics-certs\") pod \"network-metrics-daemon-9jcw5\" (UID: \"8ee8bd80-e264-413e-84d3-1a40c8c412ca\") " pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:09:59.470239 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:59.470213    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s8wbv\""
Jul 15 12:09:59.478267 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:59.478252    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jcw5"
Jul 15 12:09:59.649206 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:09:59.649175    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jcw5"]
Jul 15 12:09:59.653961 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:09:59.653921    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee8bd80_e264_413e_84d3_1a40c8c412ca.slice/crio-b355582edee7997dcb43152b8b2965f8b8b125fa98ff67887c1cce406243fa5d WatchSource:0}: Error finding container b355582edee7997dcb43152b8b2965f8b8b125fa98ff67887c1cce406243fa5d: Status 404 returned error can't find the container with id b355582edee7997dcb43152b8b2965f8b8b125fa98ff67887c1cce406243fa5d
Jul 15 12:10:00.061111 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:00.061064    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7980fcb6-1d65-4f03-b503-3b2bea12e387" path="/var/lib/kubelet/pods/7980fcb6-1d65-4f03-b503-3b2bea12e387/volumes"
Jul 15 12:10:00.602165 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:00.602128    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jcw5" event={"ID":"8ee8bd80-e264-413e-84d3-1a40c8c412ca","Type":"ContainerStarted","Data":"b355582edee7997dcb43152b8b2965f8b8b125fa98ff67887c1cce406243fa5d"}
Jul 15 12:10:01.611540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:01.611504    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jcw5" event={"ID":"8ee8bd80-e264-413e-84d3-1a40c8c412ca","Type":"ContainerStarted","Data":"30a63eb7fa294633ec9be76951047f09646a36e512690826e5f6bc9946ec1011"}
Jul 15 12:10:02.616190 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:02.616144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jcw5" event={"ID":"8ee8bd80-e264-413e-84d3-1a40c8c412ca","Type":"ContainerStarted","Data":"1f20dd89ff53c43867bea10d20aee4c921752209603a9d2c7b40b13f1f03a280"}
Jul 15 12:10:02.634502 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:02.634452    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9jcw5" podStartSLOduration=130.824675577 podStartE2EDuration="2m12.634437357s" totalImagesPullingTime="1.80976178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:10:02.634013872 +0000 UTC m=+133.733963212" watchObservedRunningTime="2026-07-15 12:10:02.634437357 +0000 UTC m=+133.734386694"
Jul 15 12:10:46.115696 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.115618    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-f6c7669b5-wdznb"]
Jul 15 12:10:46.116298 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.115850    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7980fcb6-1d65-4f03-b503-3b2bea12e387" containerName="registry"
Jul 15 12:10:46.116298 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.115860    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7980fcb6-1d65-4f03-b503-3b2bea12e387" containerName="registry"
Jul 15 12:10:46.116298 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.115914    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7980fcb6-1d65-4f03-b503-3b2bea12e387" containerName="registry"
Jul 15 12:10:46.117824 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.117807    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.120847 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.120824    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 15 12:10:46.121384 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.121364    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 15 12:10:46.121496 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.121407    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 15 12:10:46.121611 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.121598    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lrd8n\""
Jul 15 12:10:46.121762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.121752    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 15 12:10:46.121949 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.121929    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 15 12:10:46.127149 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.127130    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 15 12:10:46.141924 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.141901    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f6c7669b5-wdznb"]
Jul 15 12:10:46.220696 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.220664    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-telemeter-client-tls\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.220696 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.220698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-federate-client-tls\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.220894 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.220716    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.220894 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.220809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77kl5\" (UniqueName: \"kubernetes.io/projected/8becae98-30ea-46f9-a94d-9114cdec1469-kube-api-access-77kl5\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.220957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.220927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.220992 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.220979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-serving-certs-ca-bundle\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.221026 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.221007    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-secret-telemeter-client\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.221057 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.221043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-metrics-client-ca\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322354 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322318    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77kl5\" (UniqueName: \"kubernetes.io/projected/8becae98-30ea-46f9-a94d-9114cdec1469-kube-api-access-77kl5\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322460 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322371    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322460 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-serving-certs-ca-bundle\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322460 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-secret-telemeter-client\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322460 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-metrics-client-ca\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322597 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-telemeter-client-tls\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322597 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-federate-client-tls\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.322734 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.322709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.323272 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.323245    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-serving-certs-ca-bundle\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.323463 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.323264    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-metrics-client-ca\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.323588 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.323503    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8becae98-30ea-46f9-a94d-9114cdec1469-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.325125 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.325082    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-telemeter-client-tls\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.325235 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.325179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-federate-client-tls\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.325411 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.325394    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-secret-telemeter-client\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.325479 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.325452    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8becae98-30ea-46f9-a94d-9114cdec1469-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.332803 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.332783    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77kl5\" (UniqueName: \"kubernetes.io/projected/8becae98-30ea-46f9-a94d-9114cdec1469-kube-api-access-77kl5\") pod \"telemeter-client-f6c7669b5-wdznb\" (UID: \"8becae98-30ea-46f9-a94d-9114cdec1469\") " pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.427579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.427505    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb"
Jul 15 12:10:46.560633 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.560604    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f6c7669b5-wdznb"]
Jul 15 12:10:46.563822 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:10:46.563789    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8becae98_30ea_46f9_a94d_9114cdec1469.slice/crio-2f2b7901f4aba30cc20ecaa9937ab2ab2f2945e5d6850290ef65d263aedfa085 WatchSource:0}: Error finding container 2f2b7901f4aba30cc20ecaa9937ab2ab2f2945e5d6850290ef65d263aedfa085: Status 404 returned error can't find the container with id 2f2b7901f4aba30cc20ecaa9937ab2ab2f2945e5d6850290ef65d263aedfa085
Jul 15 12:10:46.772578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:46.772543    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb" event={"ID":"8becae98-30ea-46f9-a94d-9114cdec1469","Type":"ContainerStarted","Data":"2f2b7901f4aba30cc20ecaa9937ab2ab2f2945e5d6850290ef65d263aedfa085"}
Jul 15 12:10:48.780719 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:48.780684    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb" event={"ID":"8becae98-30ea-46f9-a94d-9114cdec1469","Type":"ContainerStarted","Data":"f07da84fac75541ce0cbe44c241e1cc8b6123b68a6736a7481366977b02470fd"}
Jul 15 12:10:49.786059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:49.786022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb" event={"ID":"8becae98-30ea-46f9-a94d-9114cdec1469","Type":"ContainerStarted","Data":"00c9fb279c35ae7d179db52b38aeb6de71337bd13a7927923fa91523b5733a7b"}
Jul 15 12:10:49.786392 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:49.786104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb" event={"ID":"8becae98-30ea-46f9-a94d-9114cdec1469","Type":"ContainerStarted","Data":"baa8ddc577c2223503ae261f29bc9381ab16b7d74b9956e2266658f30ee26800"}
Jul 15 12:10:49.816170 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:10:49.816123    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-f6c7669b5-wdznb" podStartSLOduration=0.854636324 podStartE2EDuration="3.816107003s" totalImagesPullingTime="2.961470679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:10:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:10:49.813252788 +0000 UTC m=+180.913202135" watchObservedRunningTime="2026-07-15 12:10:49.816107003 +0000 UTC m=+180.916056332"
Jul 15 12:12:49.332186 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:12:49.332157    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:12:49.332694 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:12:49.332157    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:12:49.340742 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:12:49.340716    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 12:13:26.000728 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.000692    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"]
Jul 15 12:13:26.003004 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.002984    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.005397 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.005375    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 12:13:26.005788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.005771    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 12:13:26.006170 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.006154    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d22hx\""
Jul 15 12:13:26.013149 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.013124    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"]
Jul 15 12:13:26.063527 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.063494    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4ssm\" (UniqueName: \"kubernetes.io/projected/e9196083-56b4-4a48-ab01-1edbc43110ec-kube-api-access-k4ssm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.063703 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.063533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.063703 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.063554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.163880 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.163841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.164022 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.163882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.164022 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.163949    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4ssm\" (UniqueName: \"kubernetes.io/projected/e9196083-56b4-4a48-ab01-1edbc43110ec-kube-api-access-k4ssm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.164343 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.164320    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.164412 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.164380    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.172822 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.172799    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4ssm\" (UniqueName: \"kubernetes.io/projected/e9196083-56b4-4a48-ab01-1edbc43110ec-kube-api-access-k4ssm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.312467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.312377    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:26.447507 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.447473    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"]
Jul 15 12:13:26.450619 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:13:26.450588    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9196083_56b4_4a48_ab01_1edbc43110ec.slice/crio-d432538291d90e684b2ac5228739a3c1bf41508cfab4fb9ea0bac7af0e4cf8fb WatchSource:0}: Error finding container d432538291d90e684b2ac5228739a3c1bf41508cfab4fb9ea0bac7af0e4cf8fb: Status 404 returned error can't find the container with id d432538291d90e684b2ac5228739a3c1bf41508cfab4fb9ea0bac7af0e4cf8fb
Jul 15 12:13:26.452494 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:26.452474    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:13:27.305403 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:27.305367    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r" event={"ID":"e9196083-56b4-4a48-ab01-1edbc43110ec","Type":"ContainerStarted","Data":"d432538291d90e684b2ac5228739a3c1bf41508cfab4fb9ea0bac7af0e4cf8fb"}
Jul 15 12:13:34.336992 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:34.336898    2582 generic.go:356] "Generic (PLEG): container finished" podID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerID="42addfe515330e09ec6552bd584931d0aecd74dc3ed5e8aa188d1decbe1dcfa9" exitCode=0
Jul 15 12:13:34.337426 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:34.336986    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r" event={"ID":"e9196083-56b4-4a48-ab01-1edbc43110ec","Type":"ContainerDied","Data":"42addfe515330e09ec6552bd584931d0aecd74dc3ed5e8aa188d1decbe1dcfa9"}
Jul 15 12:13:39.358601 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:39.358563    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r" event={"ID":"e9196083-56b4-4a48-ab01-1edbc43110ec","Type":"ContainerStarted","Data":"0192e08941ce01e56e7cd4f11e72e53960e7602bbec259567e79f6e398e75afb"}
Jul 15 12:13:40.363181 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:40.363147    2582 generic.go:356] "Generic (PLEG): container finished" podID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerID="0192e08941ce01e56e7cd4f11e72e53960e7602bbec259567e79f6e398e75afb" exitCode=0
Jul 15 12:13:40.363570 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:40.363207    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r" event={"ID":"e9196083-56b4-4a48-ab01-1edbc43110ec","Type":"ContainerDied","Data":"0192e08941ce01e56e7cd4f11e72e53960e7602bbec259567e79f6e398e75afb"}
Jul 15 12:13:50.403489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:50.403447    2582 generic.go:356] "Generic (PLEG): container finished" podID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerID="1be322374146c613961d4985c59f13c7eeaf41af3a4818fcd1b15b6a1c1eb8d7" exitCode=0
Jul 15 12:13:50.403489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:50.403496    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r" event={"ID":"e9196083-56b4-4a48-ab01-1edbc43110ec","Type":"ContainerDied","Data":"1be322374146c613961d4985c59f13c7eeaf41af3a4818fcd1b15b6a1c1eb8d7"}
Jul 15 12:13:51.529982 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.529957    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:51.662296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.662202    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4ssm\" (UniqueName: \"kubernetes.io/projected/e9196083-56b4-4a48-ab01-1edbc43110ec-kube-api-access-k4ssm\") pod \"e9196083-56b4-4a48-ab01-1edbc43110ec\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") "
Jul 15 12:13:51.662296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.662291    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-util\") pod \"e9196083-56b4-4a48-ab01-1edbc43110ec\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") "
Jul 15 12:13:51.662496 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.662448    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-bundle\") pod \"e9196083-56b4-4a48-ab01-1edbc43110ec\" (UID: \"e9196083-56b4-4a48-ab01-1edbc43110ec\") "
Jul 15 12:13:51.663057 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.663033    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-bundle" (OuterVolumeSpecName: "bundle") pod "e9196083-56b4-4a48-ab01-1edbc43110ec" (UID: "e9196083-56b4-4a48-ab01-1edbc43110ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:13:51.664447 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.664425    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9196083-56b4-4a48-ab01-1edbc43110ec-kube-api-access-k4ssm" (OuterVolumeSpecName: "kube-api-access-k4ssm") pod "e9196083-56b4-4a48-ab01-1edbc43110ec" (UID: "e9196083-56b4-4a48-ab01-1edbc43110ec"). InnerVolumeSpecName "kube-api-access-k4ssm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:13:51.666234 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.666204    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-util" (OuterVolumeSpecName: "util") pod "e9196083-56b4-4a48-ab01-1edbc43110ec" (UID: "e9196083-56b4-4a48-ab01-1edbc43110ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:13:51.763782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.763747    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-util\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:13:51.763782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.763775    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9196083-56b4-4a48-ab01-1edbc43110ec-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:13:51.763782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:51.763786    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4ssm\" (UniqueName: \"kubernetes.io/projected/e9196083-56b4-4a48-ab01-1edbc43110ec-kube-api-access-k4ssm\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:13:52.412175 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:52.412076    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r"
Jul 15 12:13:52.412175 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:52.412085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xbv6r" event={"ID":"e9196083-56b4-4a48-ab01-1edbc43110ec","Type":"ContainerDied","Data":"d432538291d90e684b2ac5228739a3c1bf41508cfab4fb9ea0bac7af0e4cf8fb"}
Jul 15 12:13:52.412175 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:52.412157    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d432538291d90e684b2ac5228739a3c1bf41508cfab4fb9ea0bac7af0e4cf8fb"
Jul 15 12:13:59.194939 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.194901    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"]
Jul 15 12:13:59.195467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.195284    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerName="extract"
Jul 15 12:13:59.195467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.195303    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerName="extract"
Jul 15 12:13:59.195467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.195323    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerName="pull"
Jul 15 12:13:59.195467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.195331    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerName="pull"
Jul 15 12:13:59.195467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.195357    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerName="util"
Jul 15 12:13:59.195467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.195364    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerName="util"
Jul 15 12:13:59.195467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.195429    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9196083-56b4-4a48-ab01-1edbc43110ec" containerName="extract"
Jul 15 12:13:59.197422 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.197401    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.200890 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.200865    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ttzm6\""
Jul 15 12:13:59.201029 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.200918    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:13:59.201029 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.200866    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 15 12:13:59.215684 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.215650    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"]
Jul 15 12:13:59.319416 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.319361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvrdq\" (UniqueName: \"kubernetes.io/projected/60648e0b-52f7-488d-bb23-85bc9f912e56-kube-api-access-tvrdq\") pod \"cert-manager-operator-controller-manager-55c444bf86-rtmhk\" (UID: \"60648e0b-52f7-488d-bb23-85bc9f912e56\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.319595 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.319473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60648e0b-52f7-488d-bb23-85bc9f912e56-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rtmhk\" (UID: \"60648e0b-52f7-488d-bb23-85bc9f912e56\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.420152 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.420088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60648e0b-52f7-488d-bb23-85bc9f912e56-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rtmhk\" (UID: \"60648e0b-52f7-488d-bb23-85bc9f912e56\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.420318 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.420165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvrdq\" (UniqueName: \"kubernetes.io/projected/60648e0b-52f7-488d-bb23-85bc9f912e56-kube-api-access-tvrdq\") pod \"cert-manager-operator-controller-manager-55c444bf86-rtmhk\" (UID: \"60648e0b-52f7-488d-bb23-85bc9f912e56\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.420479 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.420459    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60648e0b-52f7-488d-bb23-85bc9f912e56-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rtmhk\" (UID: \"60648e0b-52f7-488d-bb23-85bc9f912e56\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.437202 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.437170    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvrdq\" (UniqueName: \"kubernetes.io/projected/60648e0b-52f7-488d-bb23-85bc9f912e56-kube-api-access-tvrdq\") pod \"cert-manager-operator-controller-manager-55c444bf86-rtmhk\" (UID: \"60648e0b-52f7-488d-bb23-85bc9f912e56\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.506155 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.506111    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"
Jul 15 12:13:59.642892 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:13:59.642855    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk"]
Jul 15 12:13:59.645973 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:13:59.645939    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60648e0b_52f7_488d_bb23_85bc9f912e56.slice/crio-60ea30c4d0b06bae27f9a8a026c03eb094e0dfb8ced98c814a8306865d7b2d27 WatchSource:0}: Error finding container 60ea30c4d0b06bae27f9a8a026c03eb094e0dfb8ced98c814a8306865d7b2d27: Status 404 returned error can't find the container with id 60ea30c4d0b06bae27f9a8a026c03eb094e0dfb8ced98c814a8306865d7b2d27
Jul 15 12:14:00.440131 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:00.440078    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk" event={"ID":"60648e0b-52f7-488d-bb23-85bc9f912e56","Type":"ContainerStarted","Data":"60ea30c4d0b06bae27f9a8a026c03eb094e0dfb8ced98c814a8306865d7b2d27"}
Jul 15 12:14:03.455875 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:03.455841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk" event={"ID":"60648e0b-52f7-488d-bb23-85bc9f912e56","Type":"ContainerStarted","Data":"d896c8f4da4645e8f9fb2bfdf754d36f9b6b2c91e166bb5df239cfc8dbff5fd5"}
Jul 15 12:14:03.480323 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:03.480263    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rtmhk" podStartSLOduration=1.733905151 podStartE2EDuration="4.480245032s" totalImagesPullingTime="2.746339881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:14:03.4787667 +0000 UTC m=+374.578716041" watchObservedRunningTime="2026-07-15 12:14:03.480245032 +0000 UTC m=+374.580194371"
Jul 15 12:14:04.067123 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.067079    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"]
Jul 15 12:14:04.069619 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.069601    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.074407 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.074380    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 12:14:04.074407 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.074400    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d22hx\""
Jul 15 12:14:04.074603 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.074380    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 12:14:04.086065 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.086034    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"]
Jul 15 12:14:04.160600 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.160567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gzqf\" (UniqueName: \"kubernetes.io/projected/18d40e2b-6614-4950-ae0e-670816923f88-kube-api-access-5gzqf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.160788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.160617    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.160788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.160682    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.261950 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.261911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.262146 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.261967    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gzqf\" (UniqueName: \"kubernetes.io/projected/18d40e2b-6614-4950-ae0e-670816923f88-kube-api-access-5gzqf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.262146 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.262003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.262341 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.262314    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.262374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.262348    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.292193 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.292160    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gzqf\" (UniqueName: \"kubernetes.io/projected/18d40e2b-6614-4950-ae0e-670816923f88-kube-api-access-5gzqf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.378848 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.378769    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:04.508670 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:04.508640    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"]
Jul 15 12:14:04.511378 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:14:04.511351    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d40e2b_6614_4950_ae0e_670816923f88.slice/crio-4ab3e04390a586e1edce61842f2eb2e5cd3c8bda0f3faa8fc4586486b8890894 WatchSource:0}: Error finding container 4ab3e04390a586e1edce61842f2eb2e5cd3c8bda0f3faa8fc4586486b8890894: Status 404 returned error can't find the container with id 4ab3e04390a586e1edce61842f2eb2e5cd3c8bda0f3faa8fc4586486b8890894
Jul 15 12:14:05.466128 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:05.466076    2582 generic.go:356] "Generic (PLEG): container finished" podID="18d40e2b-6614-4950-ae0e-670816923f88" containerID="1c4e0b04b187acf274b73c9f5ff816ed1b9375d8a2ee10e9c59f326c072bb222" exitCode=0
Jul 15 12:14:05.466128 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:05.466123    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf" event={"ID":"18d40e2b-6614-4950-ae0e-670816923f88","Type":"ContainerDied","Data":"1c4e0b04b187acf274b73c9f5ff816ed1b9375d8a2ee10e9c59f326c072bb222"}
Jul 15 12:14:05.466320 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:05.466174    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf" event={"ID":"18d40e2b-6614-4950-ae0e-670816923f88","Type":"ContainerStarted","Data":"4ab3e04390a586e1edce61842f2eb2e5cd3c8bda0f3faa8fc4586486b8890894"}
Jul 15 12:14:06.502298 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.502260    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"]
Jul 15 12:14:06.504622 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.504595    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:06.507251 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.507226    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 12:14:06.507399 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.507312    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-67m99\""
Jul 15 12:14:06.507473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.507458    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 12:14:06.516751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.516728    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"]
Jul 15 12:14:06.680019 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.679984    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq8pw\" (UniqueName: \"kubernetes.io/projected/a2fdd888-5557-453f-8c24-c07b17ede95f-kube-api-access-fq8pw\") pod \"cert-manager-cainjector-559cdd8c87-lrsj8\" (UID: \"a2fdd888-5557-453f-8c24-c07b17ede95f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:06.680207 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.680029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2fdd888-5557-453f-8c24-c07b17ede95f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lrsj8\" (UID: \"a2fdd888-5557-453f-8c24-c07b17ede95f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:06.780885 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.780790    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq8pw\" (UniqueName: \"kubernetes.io/projected/a2fdd888-5557-453f-8c24-c07b17ede95f-kube-api-access-fq8pw\") pod \"cert-manager-cainjector-559cdd8c87-lrsj8\" (UID: \"a2fdd888-5557-453f-8c24-c07b17ede95f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:06.780885 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.780859    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2fdd888-5557-453f-8c24-c07b17ede95f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lrsj8\" (UID: \"a2fdd888-5557-453f-8c24-c07b17ede95f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:06.789406 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.789377    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2fdd888-5557-453f-8c24-c07b17ede95f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lrsj8\" (UID: \"a2fdd888-5557-453f-8c24-c07b17ede95f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:06.789552 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.789522    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq8pw\" (UniqueName: \"kubernetes.io/projected/a2fdd888-5557-453f-8c24-c07b17ede95f-kube-api-access-fq8pw\") pod \"cert-manager-cainjector-559cdd8c87-lrsj8\" (UID: \"a2fdd888-5557-453f-8c24-c07b17ede95f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:06.815498 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:06.815463    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"
Jul 15 12:14:07.430166 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:07.430110    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8"]
Jul 15 12:14:07.437214 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:14:07.437184    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2fdd888_5557_453f_8c24_c07b17ede95f.slice/crio-1d3fa0a83d35a206dc74bd82182e2e8f91ca88fedc7fa4edecc41c0059c45ecf WatchSource:0}: Error finding container 1d3fa0a83d35a206dc74bd82182e2e8f91ca88fedc7fa4edecc41c0059c45ecf: Status 404 returned error can't find the container with id 1d3fa0a83d35a206dc74bd82182e2e8f91ca88fedc7fa4edecc41c0059c45ecf
Jul 15 12:14:07.476550 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:07.476516    2582 generic.go:356] "Generic (PLEG): container finished" podID="18d40e2b-6614-4950-ae0e-670816923f88" containerID="e3e2c624006e9fb1aec0577ed3f2b7d7a517bc7604133a23710188e936973ccb" exitCode=0
Jul 15 12:14:07.476731 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:07.476601    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf" event={"ID":"18d40e2b-6614-4950-ae0e-670816923f88","Type":"ContainerDied","Data":"e3e2c624006e9fb1aec0577ed3f2b7d7a517bc7604133a23710188e936973ccb"}
Jul 15 12:14:07.477823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:07.477799    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8" event={"ID":"a2fdd888-5557-453f-8c24-c07b17ede95f","Type":"ContainerStarted","Data":"1d3fa0a83d35a206dc74bd82182e2e8f91ca88fedc7fa4edecc41c0059c45ecf"}
Jul 15 12:14:08.484607 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:08.484567    2582 generic.go:356] "Generic (PLEG): container finished" podID="18d40e2b-6614-4950-ae0e-670816923f88" containerID="5a427bc7e6535f932537d19b7028c8b0d54a7f8ced78878fa3b86d2785058423" exitCode=0
Jul 15 12:14:08.485067 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:08.484641    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf" event={"ID":"18d40e2b-6614-4950-ae0e-670816923f88","Type":"ContainerDied","Data":"5a427bc7e6535f932537d19b7028c8b0d54a7f8ced78878fa3b86d2785058423"}
Jul 15 12:14:10.094657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.094629    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:10.206931 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.206909    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-bundle\") pod \"18d40e2b-6614-4950-ae0e-670816923f88\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") "
Jul 15 12:14:10.207033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.206956    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gzqf\" (UniqueName: \"kubernetes.io/projected/18d40e2b-6614-4950-ae0e-670816923f88-kube-api-access-5gzqf\") pod \"18d40e2b-6614-4950-ae0e-670816923f88\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") "
Jul 15 12:14:10.207033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.206978    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-util\") pod \"18d40e2b-6614-4950-ae0e-670816923f88\" (UID: \"18d40e2b-6614-4950-ae0e-670816923f88\") "
Jul 15 12:14:10.207317 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.207268    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-bundle" (OuterVolumeSpecName: "bundle") pod "18d40e2b-6614-4950-ae0e-670816923f88" (UID: "18d40e2b-6614-4950-ae0e-670816923f88"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:14:10.208974 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.208949    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d40e2b-6614-4950-ae0e-670816923f88-kube-api-access-5gzqf" (OuterVolumeSpecName: "kube-api-access-5gzqf") pod "18d40e2b-6614-4950-ae0e-670816923f88" (UID: "18d40e2b-6614-4950-ae0e-670816923f88"). InnerVolumeSpecName "kube-api-access-5gzqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:14:10.211642 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.211614    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-util" (OuterVolumeSpecName: "util") pod "18d40e2b-6614-4950-ae0e-670816923f88" (UID: "18d40e2b-6614-4950-ae0e-670816923f88"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:14:10.307932 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.307896    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:14:10.307932 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.307926    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gzqf\" (UniqueName: \"kubernetes.io/projected/18d40e2b-6614-4950-ae0e-670816923f88-kube-api-access-5gzqf\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:14:10.307932 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.307936    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d40e2b-6614-4950-ae0e-670816923f88-util\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:14:10.495242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.495208    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf" event={"ID":"18d40e2b-6614-4950-ae0e-670816923f88","Type":"ContainerDied","Data":"4ab3e04390a586e1edce61842f2eb2e5cd3c8bda0f3faa8fc4586486b8890894"}
Jul 15 12:14:10.495242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.495238    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffwnnf"
Jul 15 12:14:10.495489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.495259    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ab3e04390a586e1edce61842f2eb2e5cd3c8bda0f3faa8fc4586486b8890894"
Jul 15 12:14:10.496673 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.496637    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8" event={"ID":"a2fdd888-5557-453f-8c24-c07b17ede95f","Type":"ContainerStarted","Data":"adf2b40555ffb2bf56ff0048fe724c7977151fafa9114146f173d0662e0bcee1"}
Jul 15 12:14:10.515782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:10.515734    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lrsj8" podStartSLOduration=1.808889979 podStartE2EDuration="4.51571737s" totalImagesPullingTime="2.706827391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:14:10.513939843 +0000 UTC m=+381.613889182" watchObservedRunningTime="2026-07-15 12:14:10.51571737 +0000 UTC m=+381.615666708"
Jul 15 12:14:11.649261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649222    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j4wmv"]
Jul 15 12:14:11.649627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649478    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d40e2b-6614-4950-ae0e-670816923f88" containerName="pull"
Jul 15 12:14:11.649627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649488    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d40e2b-6614-4950-ae0e-670816923f88" containerName="pull"
Jul 15 12:14:11.649627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649506    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d40e2b-6614-4950-ae0e-670816923f88" containerName="util"
Jul 15 12:14:11.649627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649511    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d40e2b-6614-4950-ae0e-670816923f88" containerName="util"
Jul 15 12:14:11.649627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649517    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d40e2b-6614-4950-ae0e-670816923f88" containerName="extract"
Jul 15 12:14:11.649627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649523    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d40e2b-6614-4950-ae0e-670816923f88" containerName="extract"
Jul 15 12:14:11.649627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.649578    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d40e2b-6614-4950-ae0e-670816923f88" containerName="extract"
Jul 15 12:14:11.652992 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.652973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:11.655609 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.655590    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qn478\""
Jul 15 12:14:11.666304 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.666281    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j4wmv"]
Jul 15 12:14:11.820325 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.820284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5bd9862-cd9b-4f98-b13a-03fc6bf04f89-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j4wmv\" (UID: \"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:11.820501 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.820336    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7xgk\" (UniqueName: \"kubernetes.io/projected/c5bd9862-cd9b-4f98-b13a-03fc6bf04f89-kube-api-access-x7xgk\") pod \"cert-manager-webhook-6486ffd997-j4wmv\" (UID: \"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:11.921144 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.921027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7xgk\" (UniqueName: \"kubernetes.io/projected/c5bd9862-cd9b-4f98-b13a-03fc6bf04f89-kube-api-access-x7xgk\") pod \"cert-manager-webhook-6486ffd997-j4wmv\" (UID: \"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:11.921144 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.921110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5bd9862-cd9b-4f98-b13a-03fc6bf04f89-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j4wmv\" (UID: \"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:11.930782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.930744    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7xgk\" (UniqueName: \"kubernetes.io/projected/c5bd9862-cd9b-4f98-b13a-03fc6bf04f89-kube-api-access-x7xgk\") pod \"cert-manager-webhook-6486ffd997-j4wmv\" (UID: \"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:11.930923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.930903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5bd9862-cd9b-4f98-b13a-03fc6bf04f89-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j4wmv\" (UID: \"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:11.962036 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:11.961996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:12.091035 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:12.091003    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j4wmv"]
Jul 15 12:14:12.093831 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:14:12.093800    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5bd9862_cd9b_4f98_b13a_03fc6bf04f89.slice/crio-bc06d00a18acf2406a9953effcd44e27fd572ac20f47f5c67c61b55bf47c7d64 WatchSource:0}: Error finding container bc06d00a18acf2406a9953effcd44e27fd572ac20f47f5c67c61b55bf47c7d64: Status 404 returned error can't find the container with id bc06d00a18acf2406a9953effcd44e27fd572ac20f47f5c67c61b55bf47c7d64
Jul 15 12:14:12.506598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:12.506559    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv" event={"ID":"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89","Type":"ContainerStarted","Data":"8046835d7588be986a6eb862ae7ebf413cfc024bca88e024ff6632dd429431bd"}
Jul 15 12:14:12.506598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:12.506600    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv" event={"ID":"c5bd9862-cd9b-4f98-b13a-03fc6bf04f89","Type":"ContainerStarted","Data":"bc06d00a18acf2406a9953effcd44e27fd572ac20f47f5c67c61b55bf47c7d64"}
Jul 15 12:14:12.506862 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:12.506634    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:18.513077 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:18.513048    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv"
Jul 15 12:14:18.537428 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:18.537370    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-j4wmv" podStartSLOduration=7.537353612 podStartE2EDuration="7.537353612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:14:12.525400493 +0000 UTC m=+383.625349871" watchObservedRunningTime="2026-07-15 12:14:18.537353612 +0000 UTC m=+389.637302950"
Jul 15 12:14:25.905473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:25.905433    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"]
Jul 15 12:14:25.909861 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:25.909843    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:25.912123 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:25.912082    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 12:14:25.912258 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:25.912157    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 12:14:25.912846 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:25.912831    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d22hx\""
Jul 15 12:14:25.920447 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:25.920422    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"]
Jul 15 12:14:26.035322 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.035282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.035489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.035332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.035489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.035416    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5zhd\" (UniqueName: \"kubernetes.io/projected/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-kube-api-access-m5zhd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.136748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.136710    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.136914 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.136774    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5zhd\" (UniqueName: \"kubernetes.io/projected/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-kube-api-access-m5zhd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.136914 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.136809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.137183 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.137168    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.137241 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.137222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.146147 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.146114    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5zhd\" (UniqueName: \"kubernetes.io/projected/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-kube-api-access-m5zhd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.220018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.219983    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:26.372561 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.372535    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"]
Jul 15 12:14:26.375396 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:14:26.375355    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960dd9e4_a82d_4261_80b8_de1a6ff00ab7.slice/crio-a2eced24a36613281d804a3a28830a888f40a84ddbefb6e69bcc849cfeee6d7b WatchSource:0}: Error finding container a2eced24a36613281d804a3a28830a888f40a84ddbefb6e69bcc849cfeee6d7b: Status 404 returned error can't find the container with id a2eced24a36613281d804a3a28830a888f40a84ddbefb6e69bcc849cfeee6d7b
Jul 15 12:14:26.561819 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.561785    2582 generic.go:356] "Generic (PLEG): container finished" podID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerID="10b67138789d2cf372a6b643a3b4832c08d3ca83094f1d3e14d5922aef3afdca" exitCode=0
Jul 15 12:14:26.562004 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.561885    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p" event={"ID":"960dd9e4-a82d-4261-80b8-de1a6ff00ab7","Type":"ContainerDied","Data":"10b67138789d2cf372a6b643a3b4832c08d3ca83094f1d3e14d5922aef3afdca"}
Jul 15 12:14:26.562004 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:26.561940    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p" event={"ID":"960dd9e4-a82d-4261-80b8-de1a6ff00ab7","Type":"ContainerStarted","Data":"a2eced24a36613281d804a3a28830a888f40a84ddbefb6e69bcc849cfeee6d7b"}
Jul 15 12:14:27.568314 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:27.568274    2582 generic.go:356] "Generic (PLEG): container finished" podID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerID="1562d1b14d53413db9d48b164fb4751615d956537b9fd448e15764723af642db" exitCode=0
Jul 15 12:14:27.568753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:27.568345    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p" event={"ID":"960dd9e4-a82d-4261-80b8-de1a6ff00ab7","Type":"ContainerDied","Data":"1562d1b14d53413db9d48b164fb4751615d956537b9fd448e15764723af642db"}
Jul 15 12:14:28.575652 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:28.575613    2582 generic.go:356] "Generic (PLEG): container finished" podID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerID="9257344eae49f0b3fbd621e367c9640fb11bf294a9c5e79e04ad85212583aaaf" exitCode=0
Jul 15 12:14:28.576027 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:28.575656    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p" event={"ID":"960dd9e4-a82d-4261-80b8-de1a6ff00ab7","Type":"ContainerDied","Data":"9257344eae49f0b3fbd621e367c9640fb11bf294a9c5e79e04ad85212583aaaf"}
Jul 15 12:14:29.711399 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.711374    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:29.762683 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.762651    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5zhd\" (UniqueName: \"kubernetes.io/projected/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-kube-api-access-m5zhd\") pod \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") "
Jul 15 12:14:29.762683 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.762689    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-util\") pod \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") "
Jul 15 12:14:29.762898 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.762724    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-bundle\") pod \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\" (UID: \"960dd9e4-a82d-4261-80b8-de1a6ff00ab7\") "
Jul 15 12:14:29.763462 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.763440    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-bundle" (OuterVolumeSpecName: "bundle") pod "960dd9e4-a82d-4261-80b8-de1a6ff00ab7" (UID: "960dd9e4-a82d-4261-80b8-de1a6ff00ab7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:14:29.764854 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.764830    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-kube-api-access-m5zhd" (OuterVolumeSpecName: "kube-api-access-m5zhd") pod "960dd9e4-a82d-4261-80b8-de1a6ff00ab7" (UID: "960dd9e4-a82d-4261-80b8-de1a6ff00ab7"). InnerVolumeSpecName "kube-api-access-m5zhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:14:29.768360 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.768337    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-util" (OuterVolumeSpecName: "util") pod "960dd9e4-a82d-4261-80b8-de1a6ff00ab7" (UID: "960dd9e4-a82d-4261-80b8-de1a6ff00ab7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:14:29.863548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.863445    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5zhd\" (UniqueName: \"kubernetes.io/projected/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-kube-api-access-m5zhd\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:14:29.863548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.863489    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-util\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:14:29.863548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:29.863499    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960dd9e4-a82d-4261-80b8-de1a6ff00ab7-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:14:30.590868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:30.590841    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p"
Jul 15 12:14:30.591039 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:30.590841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkbz5p" event={"ID":"960dd9e4-a82d-4261-80b8-de1a6ff00ab7","Type":"ContainerDied","Data":"a2eced24a36613281d804a3a28830a888f40a84ddbefb6e69bcc849cfeee6d7b"}
Jul 15 12:14:30.591039 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:30.590973    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2eced24a36613281d804a3a28830a888f40a84ddbefb6e69bcc849cfeee6d7b"
Jul 15 12:14:42.163362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163329    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"]
Jul 15 12:14:42.163753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163607    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerName="pull"
Jul 15 12:14:42.163753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163619    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerName="pull"
Jul 15 12:14:42.163753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163640    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerName="extract"
Jul 15 12:14:42.163753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163647    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerName="extract"
Jul 15 12:14:42.163753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163653    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerName="util"
Jul 15 12:14:42.163753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163658    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerName="util"
Jul 15 12:14:42.163753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.163703    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="960dd9e4-a82d-4261-80b8-de1a6ff00ab7" containerName="extract"
Jul 15 12:14:42.165405 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.165389    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.167588 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.167562    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 12:14:42.168573 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.168554    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 12:14:42.168683 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.168604    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 12:14:42.168986 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.168972    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-j8fq9\""
Jul 15 12:14:42.169190 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.169172    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 12:14:42.184317 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.184286    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"]
Jul 15 12:14:42.259717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.259678    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c911986-3fd5-482e-b18c-102c44794762-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.259899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.259726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bj9n\" (UniqueName: \"kubernetes.io/projected/9c911986-3fd5-482e-b18c-102c44794762-kube-api-access-8bj9n\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.259899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.259810    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c911986-3fd5-482e-b18c-102c44794762-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.259899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.259853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c911986-3fd5-482e-b18c-102c44794762-manifests\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.259899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.259874    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c911986-3fd5-482e-b18c-102c44794762-tmp\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.361268 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.361233    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c911986-3fd5-482e-b18c-102c44794762-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.361268 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.361275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bj9n\" (UniqueName: \"kubernetes.io/projected/9c911986-3fd5-482e-b18c-102c44794762-kube-api-access-8bj9n\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.361470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.361299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c911986-3fd5-482e-b18c-102c44794762-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.361470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.361327    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c911986-3fd5-482e-b18c-102c44794762-manifests\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.361470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.361349    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c911986-3fd5-482e-b18c-102c44794762-tmp\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.361715 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.361696    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c911986-3fd5-482e-b18c-102c44794762-manifests\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.363661 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.363630    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c911986-3fd5-482e-b18c-102c44794762-tmp\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.363886 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.363868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c911986-3fd5-482e-b18c-102c44794762-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.363951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.363890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c911986-3fd5-482e-b18c-102c44794762-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.382468 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.382440    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bj9n\" (UniqueName: \"kubernetes.io/projected/9c911986-3fd5-482e-b18c-102c44794762-kube-api-access-8bj9n\") pod \"opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27\" (UID: \"9c911986-3fd5-482e-b18c-102c44794762\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.475612 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.475563    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:42.614786 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.614752    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"]
Jul 15 12:14:42.618718 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:14:42.618689    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c911986_3fd5_482e_b18c_102c44794762.slice/crio-eb44e8bdd55686e41020161e2994d83975eb56b59de56566c08819b9719ce761 WatchSource:0}: Error finding container eb44e8bdd55686e41020161e2994d83975eb56b59de56566c08819b9719ce761: Status 404 returned error can't find the container with id eb44e8bdd55686e41020161e2994d83975eb56b59de56566c08819b9719ce761
Jul 15 12:14:42.642181 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:42.642151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27" event={"ID":"9c911986-3fd5-482e-b18c-102c44794762","Type":"ContainerStarted","Data":"eb44e8bdd55686e41020161e2994d83975eb56b59de56566c08819b9719ce761"}
Jul 15 12:14:46.665306 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:46.665265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27" event={"ID":"9c911986-3fd5-482e-b18c-102c44794762","Type":"ContainerStarted","Data":"a4499b3008608115826e7f407efd2410784b9e2ad7add9602b5dea4bf32f3ea1"}
Jul 15 12:14:47.670725 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:47.670684    2582 generic.go:356] "Generic (PLEG): container finished" podID="9c911986-3fd5-482e-b18c-102c44794762" containerID="a4499b3008608115826e7f407efd2410784b9e2ad7add9602b5dea4bf32f3ea1" exitCode=0
Jul 15 12:14:47.671127 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:47.670751    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27" event={"ID":"9c911986-3fd5-482e-b18c-102c44794762","Type":"ContainerDied","Data":"a4499b3008608115826e7f407efd2410784b9e2ad7add9602b5dea4bf32f3ea1"}
Jul 15 12:14:48.677028 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:48.676989    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27" event={"ID":"9c911986-3fd5-482e-b18c-102c44794762","Type":"ContainerStarted","Data":"0816c71ea38a3da96f6ed19831d68f896b2539c8d1bc54a77cba2c1a416a9d01"}
Jul 15 12:14:48.677463 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:48.677151    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:14:48.702141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:48.702062    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27" podStartSLOduration=2.206799568 podStartE2EDuration="6.702044904s" totalImagesPullingTime="3.483400958s" totalInitContainerRuntime="1.011844378s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:14:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:14:48.700220599 +0000 UTC m=+419.800169938" watchObservedRunningTime="2026-07-15 12:14:48.702044904 +0000 UTC m=+419.801994242"
Jul 15 12:14:59.682687 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:14:59.682656    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27"
Jul 15 12:15:02.985956 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:02.985915    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"]
Jul 15 12:15:03.002212 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.002173    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"]
Jul 15 12:15:03.002388 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.002309    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.005599 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.005567    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 12:15:03.005762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.005567    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 12:15:03.005762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.005567    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dnz7s\""
Jul 15 12:15:03.098806 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.098769    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j2d2q"]
Jul 15 12:15:03.101024 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.101007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:03.104614 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.104590    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 15 12:15:03.104614 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.104610    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gsnp8\""
Jul 15 12:15:03.116603 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.116570    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j2d2q"]
Jul 15 12:15:03.127521 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.127490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738e511b-e0b7-46ef-8f29-56b3e5fb0584-tmp\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.127678 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.127528    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738e511b-e0b7-46ef-8f29-56b3e5fb0584-tls-certs\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.127678 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.127549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt25g\" (UniqueName: \"kubernetes.io/projected/738e511b-e0b7-46ef-8f29-56b3e5fb0584-kube-api-access-tt25g\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.229163 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.229122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcp57\" (UniqueName: \"kubernetes.io/projected/b0385147-5995-4d69-b0bd-da7ed08ca180-kube-api-access-wcp57\") pod \"odh-model-controller-7998c67dc4-j2d2q\" (UID: \"b0385147-5995-4d69-b0bd-da7ed08ca180\") " pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:03.229364 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.229182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt25g\" (UniqueName: \"kubernetes.io/projected/738e511b-e0b7-46ef-8f29-56b3e5fb0584-kube-api-access-tt25g\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.229364 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.229315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738e511b-e0b7-46ef-8f29-56b3e5fb0584-tmp\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.229364 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.229348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738e511b-e0b7-46ef-8f29-56b3e5fb0584-tls-certs\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.229531 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.229372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0385147-5995-4d69-b0bd-da7ed08ca180-cert\") pod \"odh-model-controller-7998c67dc4-j2d2q\" (UID: \"b0385147-5995-4d69-b0bd-da7ed08ca180\") " pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:03.232232 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.232194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738e511b-e0b7-46ef-8f29-56b3e5fb0584-tmp\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.232510 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.232486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738e511b-e0b7-46ef-8f29-56b3e5fb0584-tls-certs\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.244030 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.243941    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt25g\" (UniqueName: \"kubernetes.io/projected/738e511b-e0b7-46ef-8f29-56b3e5fb0584-kube-api-access-tt25g\") pod \"kube-auth-proxy-c594875d4-q5h6s\" (UID: \"738e511b-e0b7-46ef-8f29-56b3e5fb0584\") " pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.313255 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.313208    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"
Jul 15 12:15:03.330598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.330568    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0385147-5995-4d69-b0bd-da7ed08ca180-cert\") pod \"odh-model-controller-7998c67dc4-j2d2q\" (UID: \"b0385147-5995-4d69-b0bd-da7ed08ca180\") " pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:03.330795 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.330606    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcp57\" (UniqueName: \"kubernetes.io/projected/b0385147-5995-4d69-b0bd-da7ed08ca180-kube-api-access-wcp57\") pod \"odh-model-controller-7998c67dc4-j2d2q\" (UID: \"b0385147-5995-4d69-b0bd-da7ed08ca180\") " pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:03.330795 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:15:03.330753    2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 12:15:03.330918 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:15:03.330833    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0385147-5995-4d69-b0bd-da7ed08ca180-cert podName:b0385147-5995-4d69-b0bd-da7ed08ca180 nodeName:}" failed. No retries permitted until 2026-07-15 12:15:03.830810541 +0000 UTC m=+434.930759870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b0385147-5995-4d69-b0bd-da7ed08ca180-cert") pod "odh-model-controller-7998c67dc4-j2d2q" (UID: "b0385147-5995-4d69-b0bd-da7ed08ca180") : secret "odh-model-controller-webhook-cert" not found
Jul 15 12:15:03.340504 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.340473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcp57\" (UniqueName: \"kubernetes.io/projected/b0385147-5995-4d69-b0bd-da7ed08ca180-kube-api-access-wcp57\") pod \"odh-model-controller-7998c67dc4-j2d2q\" (UID: \"b0385147-5995-4d69-b0bd-da7ed08ca180\") " pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:03.456498 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.456463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c594875d4-q5h6s"]
Jul 15 12:15:03.459536 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:15:03.459504    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738e511b_e0b7_46ef_8f29_56b3e5fb0584.slice/crio-29eb7403a79bbc75dd2e0eddd231df323e4141c81ad13f1e062b6aeb60f00379 WatchSource:0}: Error finding container 29eb7403a79bbc75dd2e0eddd231df323e4141c81ad13f1e062b6aeb60f00379: Status 404 returned error can't find the container with id 29eb7403a79bbc75dd2e0eddd231df323e4141c81ad13f1e062b6aeb60f00379
Jul 15 12:15:03.741979 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.741942    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s" event={"ID":"738e511b-e0b7-46ef-8f29-56b3e5fb0584","Type":"ContainerStarted","Data":"29eb7403a79bbc75dd2e0eddd231df323e4141c81ad13f1e062b6aeb60f00379"}
Jul 15 12:15:03.834894 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.834854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0385147-5995-4d69-b0bd-da7ed08ca180-cert\") pod \"odh-model-controller-7998c67dc4-j2d2q\" (UID: \"b0385147-5995-4d69-b0bd-da7ed08ca180\") " pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:03.837381 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:03.837357    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0385147-5995-4d69-b0bd-da7ed08ca180-cert\") pod \"odh-model-controller-7998c67dc4-j2d2q\" (UID: \"b0385147-5995-4d69-b0bd-da7ed08ca180\") " pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:04.003265 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.003183    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fkrgw"]
Jul 15 12:15:04.006286 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.006267    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.010081 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.010058    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 15 12:15:04.011660 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.011641    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:04.033654 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.033620    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fkrgw"]
Jul 15 12:15:04.138204 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.138165    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6rbf\" (UniqueName: \"kubernetes.io/projected/5a8d121a-028a-40e3-9414-583ef7cf9141-kube-api-access-c6rbf\") pod \"model-serving-api-54d89bd5fc-fkrgw\" (UID: \"5a8d121a-028a-40e3-9414-583ef7cf9141\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.138361 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.138230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8d121a-028a-40e3-9414-583ef7cf9141-tls-certs\") pod \"model-serving-api-54d89bd5fc-fkrgw\" (UID: \"5a8d121a-028a-40e3-9414-583ef7cf9141\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.178180 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.178142    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j2d2q"]
Jul 15 12:15:04.178846 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:15:04.178815    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0385147_5995_4d69_b0bd_da7ed08ca180.slice/crio-f26c37e1bcc705ec9338c91f938612b0b89d7d9247969790e605d20f7dfc911d WatchSource:0}: Error finding container f26c37e1bcc705ec9338c91f938612b0b89d7d9247969790e605d20f7dfc911d: Status 404 returned error can't find the container with id f26c37e1bcc705ec9338c91f938612b0b89d7d9247969790e605d20f7dfc911d
Jul 15 12:15:04.191963 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.191926    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"]
Jul 15 12:15:04.194723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.194699    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.197336 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.197314    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 12:15:04.197452 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.197314    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 12:15:04.197927 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.197763    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 12:15:04.197927 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.197777    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 12:15:04.197927 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.197845    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vqs6l\""
Jul 15 12:15:04.216895 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.216862    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"]
Jul 15 12:15:04.239586 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.239547    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6rbf\" (UniqueName: \"kubernetes.io/projected/5a8d121a-028a-40e3-9414-583ef7cf9141-kube-api-access-c6rbf\") pod \"model-serving-api-54d89bd5fc-fkrgw\" (UID: \"5a8d121a-028a-40e3-9414-583ef7cf9141\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.239760 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.239601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8d121a-028a-40e3-9414-583ef7cf9141-tls-certs\") pod \"model-serving-api-54d89bd5fc-fkrgw\" (UID: \"5a8d121a-028a-40e3-9414-583ef7cf9141\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.243005 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.242974    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8d121a-028a-40e3-9414-583ef7cf9141-tls-certs\") pod \"model-serving-api-54d89bd5fc-fkrgw\" (UID: \"5a8d121a-028a-40e3-9414-583ef7cf9141\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.254518 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.254449    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6rbf\" (UniqueName: \"kubernetes.io/projected/5a8d121a-028a-40e3-9414-583ef7cf9141-kube-api-access-c6rbf\") pod \"model-serving-api-54d89bd5fc-fkrgw\" (UID: \"5a8d121a-028a-40e3-9414-583ef7cf9141\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.316257 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.316223    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:04.341261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.341224    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.341424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.341364    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/232d4b71-3e26-4159-9cc5-dcb43bd99e71-local-certs\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.341424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.341404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.341537 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.341455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77n6c\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-kube-api-access-77n6c\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.341537 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.341503    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-cacerts\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.341639 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.341560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.341639 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.341605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-token\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.442360 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.442329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-token\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.442548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.442384    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.442548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.442414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/232d4b71-3e26-4159-9cc5-dcb43bd99e71-local-certs\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.442548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.442439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.442548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.442477    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77n6c\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-kube-api-access-77n6c\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.442548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.442519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-cacerts\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.442805 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.442573    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.443570 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.443534    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.445333 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.445305    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/232d4b71-3e26-4159-9cc5-dcb43bd99e71-local-certs\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.445505 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.445481    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-cacerts\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.445643 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.445626    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.445924 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.445905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.460479 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.460446    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-token\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.461243 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.461217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77n6c\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-kube-api-access-77n6c\") pod \"istiod-openshift-gateway-7574459656-pfwbx\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.483804 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.483778    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fkrgw"]
Jul 15 12:15:04.486401 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:15:04.486375    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8d121a_028a_40e3_9414_583ef7cf9141.slice/crio-ab6fa563db1927b0212e625f1fd6f10f5a1ff019d29f272d4609d4194f3788c8 WatchSource:0}: Error finding container ab6fa563db1927b0212e625f1fd6f10f5a1ff019d29f272d4609d4194f3788c8: Status 404 returned error can't find the container with id ab6fa563db1927b0212e625f1fd6f10f5a1ff019d29f272d4609d4194f3788c8
Jul 15 12:15:04.505844 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.505675    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:04.659698 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.659655    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"]
Jul 15 12:15:04.663697 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:15:04.663669    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232d4b71_3e26_4159_9cc5_dcb43bd99e71.slice/crio-946666186f19d9da6f17e5ec9449e60bfcc42cfc7109a07eaae3097ece17fe56 WatchSource:0}: Error finding container 946666186f19d9da6f17e5ec9449e60bfcc42cfc7109a07eaae3097ece17fe56: Status 404 returned error can't find the container with id 946666186f19d9da6f17e5ec9449e60bfcc42cfc7109a07eaae3097ece17fe56
Jul 15 12:15:04.747653 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.747608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q" event={"ID":"b0385147-5995-4d69-b0bd-da7ed08ca180","Type":"ContainerStarted","Data":"f26c37e1bcc705ec9338c91f938612b0b89d7d9247969790e605d20f7dfc911d"}
Jul 15 12:15:04.748807 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.748782    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw" event={"ID":"5a8d121a-028a-40e3-9414-583ef7cf9141","Type":"ContainerStarted","Data":"ab6fa563db1927b0212e625f1fd6f10f5a1ff019d29f272d4609d4194f3788c8"}
Jul 15 12:15:04.749957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:04.749938    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx" event={"ID":"232d4b71-3e26-4159-9cc5-dcb43bd99e71","Type":"ContainerStarted","Data":"946666186f19d9da6f17e5ec9449e60bfcc42cfc7109a07eaae3097ece17fe56"}
Jul 15 12:15:09.376985 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.376946    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:15:09.377288 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.377038    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:15:09.780579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.780542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx" event={"ID":"232d4b71-3e26-4159-9cc5-dcb43bd99e71","Type":"ContainerStarted","Data":"862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4"}
Jul 15 12:15:09.780786 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.780723    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:09.782232 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.782205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q" event={"ID":"b0385147-5995-4d69-b0bd-da7ed08ca180","Type":"ContainerStarted","Data":"7333084b2b335ff059c8b24e09e4f9bdb712db03619ad3b0a6dc867f7fcf3bd0"}
Jul 15 12:15:09.782354 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.782332    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:09.783746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.783725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s" event={"ID":"738e511b-e0b7-46ef-8f29-56b3e5fb0584","Type":"ContainerStarted","Data":"bdeefec79eb0d8434f7959a706ed518bcff265f258506ccae094d4380fb17eb1"}
Jul 15 12:15:09.785268 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.785239    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw" event={"ID":"5a8d121a-028a-40e3-9414-583ef7cf9141","Type":"ContainerStarted","Data":"323c7271af8712b6305e0a7063ad0a386a0b20d33dd29e202855af07b5e6a869"}
Jul 15 12:15:09.785361 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.785355    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:09.816113 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.816047    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx" podStartSLOduration=1.104831439 podStartE2EDuration="5.816030835s" totalImagesPullingTime="4.711199396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:15:09.813672161 +0000 UTC m=+440.913621500" watchObservedRunningTime="2026-07-15 12:15:09.816030835 +0000 UTC m=+440.915980174"
Jul 15 12:15:09.838048 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.837983    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw" podStartSLOduration=1.95595371 podStartE2EDuration="6.837961776s" totalImagesPullingTime="4.882008066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:15:09.83557836 +0000 UTC m=+440.935527700" watchObservedRunningTime="2026-07-15 12:15:09.837961776 +0000 UTC m=+440.937911117"
Jul 15 12:15:09.857704 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.857636    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c594875d4-q5h6s" podStartSLOduration=1.942706411 podStartE2EDuration="7.857618288s" totalImagesPullingTime="5.914911877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:15:09.856428983 +0000 UTC m=+440.956378325" watchObservedRunningTime="2026-07-15 12:15:09.857618288 +0000 UTC m=+440.957567627"
Jul 15 12:15:09.890488 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:09.890431    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q" podStartSLOduration=1.6574595460000001 podStartE2EDuration="6.890413451s" totalImagesPullingTime="5.232953905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:15:09.888786349 +0000 UTC m=+440.988735690" watchObservedRunningTime="2026-07-15 12:15:09.890413451 +0000 UTC m=+440.990362790"
Jul 15 12:15:10.791702 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:10.791673    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:14.828901 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:14.828868    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"]
Jul 15 12:15:14.829401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:14.829044    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx" podUID="232d4b71-3e26-4159-9cc5-dcb43bd99e71" containerName="discovery" containerID="cri-o://862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4" gracePeriod=30
Jul 15 12:15:14.927715 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:14.927675    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"]
Jul 15 12:15:14.930318 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:14.930298    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:14.934158 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:14.934020    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dfzhl\""
Jul 15 12:15:14.939580 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:14.935537    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 15 12:15:14.948899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:14.948869    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"]
Jul 15 12:15:15.041380 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.041342    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e8d42b7-45a3-44c8-af57-6598de27bff9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g24bx\" (UID: \"8e8d42b7-45a3-44c8-af57-6598de27bff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:15.041533 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.041394    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtmkm\" (UniqueName: \"kubernetes.io/projected/8e8d42b7-45a3-44c8-af57-6598de27bff9-kube-api-access-jtmkm\") pod \"llmisvc-controller-manager-777c6bd7d5-g24bx\" (UID: \"8e8d42b7-45a3-44c8-af57-6598de27bff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:15.084464 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.084398    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:15.142721 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.142687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e8d42b7-45a3-44c8-af57-6598de27bff9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g24bx\" (UID: \"8e8d42b7-45a3-44c8-af57-6598de27bff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:15.142892 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.142735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtmkm\" (UniqueName: \"kubernetes.io/projected/8e8d42b7-45a3-44c8-af57-6598de27bff9-kube-api-access-jtmkm\") pod \"llmisvc-controller-manager-777c6bd7d5-g24bx\" (UID: \"8e8d42b7-45a3-44c8-af57-6598de27bff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:15.145300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.145278    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e8d42b7-45a3-44c8-af57-6598de27bff9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-g24bx\" (UID: \"8e8d42b7-45a3-44c8-af57-6598de27bff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:15.154027 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.154001    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtmkm\" (UniqueName: \"kubernetes.io/projected/8e8d42b7-45a3-44c8-af57-6598de27bff9-kube-api-access-jtmkm\") pod \"llmisvc-controller-manager-777c6bd7d5-g24bx\" (UID: \"8e8d42b7-45a3-44c8-af57-6598de27bff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:15.243921 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.243874    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/232d4b71-3e26-4159-9cc5-dcb43bd99e71-local-certs\") pod \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") "
Jul 15 12:15:15.244115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.243952    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-ca-configmap\") pod \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") "
Jul 15 12:15:15.244115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.243966    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:15.244115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.243987    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-token\") pod \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") "
Jul 15 12:15:15.244115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.244026    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-kubeconfig\") pod \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") "
Jul 15 12:15:15.244115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.244051    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-cacerts\") pod \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") "
Jul 15 12:15:15.244408 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.244381    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77n6c\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-kube-api-access-77n6c\") pod \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") "
Jul 15 12:15:15.244555 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.244445    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "232d4b71-3e26-4159-9cc5-dcb43bd99e71" (UID: "232d4b71-3e26-4159-9cc5-dcb43bd99e71"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:15:15.244555 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.244450    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-dns-cert\") pod \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\" (UID: \"232d4b71-3e26-4159-9cc5-dcb43bd99e71\") "
Jul 15 12:15:15.244762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.244740    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-ca-configmap\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:15:15.246987 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.246953    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "232d4b71-3e26-4159-9cc5-dcb43bd99e71" (UID: "232d4b71-3e26-4159-9cc5-dcb43bd99e71"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:15:15.247085 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.247011    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/232d4b71-3e26-4159-9cc5-dcb43bd99e71-local-certs" (OuterVolumeSpecName: "local-certs") pod "232d4b71-3e26-4159-9cc5-dcb43bd99e71" (UID: "232d4b71-3e26-4159-9cc5-dcb43bd99e71"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:15:15.247225 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.247201    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-cacerts" (OuterVolumeSpecName: "cacerts") pod "232d4b71-3e26-4159-9cc5-dcb43bd99e71" (UID: "232d4b71-3e26-4159-9cc5-dcb43bd99e71"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:15:15.247354 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.247335    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-kube-api-access-77n6c" (OuterVolumeSpecName: "kube-api-access-77n6c") pod "232d4b71-3e26-4159-9cc5-dcb43bd99e71" (UID: "232d4b71-3e26-4159-9cc5-dcb43bd99e71"). InnerVolumeSpecName "kube-api-access-77n6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:15:15.247401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.247380    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "232d4b71-3e26-4159-9cc5-dcb43bd99e71" (UID: "232d4b71-3e26-4159-9cc5-dcb43bd99e71"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:15:15.247445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.247392    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-token" (OuterVolumeSpecName: "istio-token") pod "232d4b71-3e26-4159-9cc5-dcb43bd99e71" (UID: "232d4b71-3e26-4159-9cc5-dcb43bd99e71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:15:15.345598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.345512    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:15:15.345598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.345542    2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-kubeconfig\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:15:15.345598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.345552    2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-cacerts\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:15:15.345598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.345562    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77n6c\" (UniqueName: \"kubernetes.io/projected/232d4b71-3e26-4159-9cc5-dcb43bd99e71-kube-api-access-77n6c\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:15:15.345598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.345573    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/232d4b71-3e26-4159-9cc5-dcb43bd99e71-istio-csr-dns-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:15:15.345598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.345582    2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/232d4b71-3e26-4159-9cc5-dcb43bd99e71-local-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:15:15.401727 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:15:15.401691    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e8d42b7_45a3_44c8_af57_6598de27bff9.slice/crio-f23b20202f0ee08fbfe2341e29789590d873ab1f4dfd72e9c04b88bc7305aabd WatchSource:0}: Error finding container f23b20202f0ee08fbfe2341e29789590d873ab1f4dfd72e9c04b88bc7305aabd: Status 404 returned error can't find the container with id f23b20202f0ee08fbfe2341e29789590d873ab1f4dfd72e9c04b88bc7305aabd
Jul 15 12:15:15.404328 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.404306    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"]
Jul 15 12:15:15.812818 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.812783    2582 generic.go:356] "Generic (PLEG): container finished" podID="232d4b71-3e26-4159-9cc5-dcb43bd99e71" containerID="862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4" exitCode=0
Jul 15 12:15:15.813014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.812835    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"
Jul 15 12:15:15.813014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.812874    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx" event={"ID":"232d4b71-3e26-4159-9cc5-dcb43bd99e71","Type":"ContainerDied","Data":"862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4"}
Jul 15 12:15:15.813014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.812930    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx" event={"ID":"232d4b71-3e26-4159-9cc5-dcb43bd99e71","Type":"ContainerDied","Data":"946666186f19d9da6f17e5ec9449e60bfcc42cfc7109a07eaae3097ece17fe56"}
Jul 15 12:15:15.813014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.812958    2582 scope.go:117] "RemoveContainer" containerID="862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4"
Jul 15 12:15:15.814213 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.814191    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx" event={"ID":"8e8d42b7-45a3-44c8-af57-6598de27bff9","Type":"ContainerStarted","Data":"f23b20202f0ee08fbfe2341e29789590d873ab1f4dfd72e9c04b88bc7305aabd"}
Jul 15 12:15:15.821870 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.821854    2582 scope.go:117] "RemoveContainer" containerID="862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4"
Jul 15 12:15:15.822206 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:15:15.822183    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4\": container with ID starting with 862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4 not found: ID does not exist" containerID="862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4"
Jul 15 12:15:15.822266 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.822215    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4"} err="failed to get container status \"862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4\": rpc error: code = NotFound desc = could not find container \"862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4\": container with ID starting with 862e440b6fa57f1ae721b92ec7e04c38ca08c5c4083e8975cd6f7e52d41d8bb4 not found: ID does not exist"
Jul 15 12:15:15.850071 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.850033    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"]
Jul 15 12:15:15.860550 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:15.860521    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfwbx"]
Jul 15 12:15:16.062332 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:16.062303    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="232d4b71-3e26-4159-9cc5-dcb43bd99e71" path="/var/lib/kubelet/pods/232d4b71-3e26-4159-9cc5-dcb43bd99e71/volumes"
Jul 15 12:15:19.835901 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:19.835859    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx" event={"ID":"8e8d42b7-45a3-44c8-af57-6598de27bff9","Type":"ContainerStarted","Data":"c660dc5a275671cf9093500a9e5caac10307638322b19fdf74cb36dc3c027541"}
Jul 15 12:15:19.836433 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:19.835984    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:15:19.861115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:19.859132    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx" podStartSLOduration=1.89444543 podStartE2EDuration="5.859110784s" totalImagesPullingTime="3.964665354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:15:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:15:19.854984124 +0000 UTC m=+450.954933464" watchObservedRunningTime="2026-07-15 12:15:19.859110784 +0000 UTC m=+450.959060119"
Jul 15 12:15:20.792199 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:20.792169    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j2d2q"
Jul 15 12:15:20.794071 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:20.794052    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fkrgw"
Jul 15 12:15:50.850065 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:15:50.850035    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-g24bx"
Jul 15 12:16:03.899269 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.899227    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2qw4q"]
Jul 15 12:16:03.899682 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.899534    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="232d4b71-3e26-4159-9cc5-dcb43bd99e71" containerName="discovery"
Jul 15 12:16:03.899682 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.899545    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="232d4b71-3e26-4159-9cc5-dcb43bd99e71" containerName="discovery"
Jul 15 12:16:03.899682 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.899621    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="232d4b71-3e26-4159-9cc5-dcb43bd99e71" containerName="discovery"
Jul 15 12:16:03.904109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.904066    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q"
Jul 15 12:16:03.906554 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.906528    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 12:16:03.907290 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.907265    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nq5jm\""
Jul 15 12:16:03.907765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.907342    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 12:16:03.910572 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:03.910546    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2qw4q"]
Jul 15 12:16:04.037549 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.037514    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mq5n\" (UniqueName: \"kubernetes.io/projected/62e2de90-103b-4c1e-8036-4de62451c46e-kube-api-access-4mq5n\") pod \"kuadrant-operator-catalog-2qw4q\" (UID: \"62e2de90-103b-4c1e-8036-4de62451c46e\") " pod="kuadrant-system/kuadrant-operator-catalog-2qw4q"
Jul 15 12:16:04.138665 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.138619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mq5n\" (UniqueName: \"kubernetes.io/projected/62e2de90-103b-4c1e-8036-4de62451c46e-kube-api-access-4mq5n\") pod \"kuadrant-operator-catalog-2qw4q\" (UID: \"62e2de90-103b-4c1e-8036-4de62451c46e\") " pod="kuadrant-system/kuadrant-operator-catalog-2qw4q"
Jul 15 12:16:04.150668 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.150604    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mq5n\" (UniqueName: \"kubernetes.io/projected/62e2de90-103b-4c1e-8036-4de62451c46e-kube-api-access-4mq5n\") pod \"kuadrant-operator-catalog-2qw4q\" (UID: \"62e2de90-103b-4c1e-8036-4de62451c46e\") " pod="kuadrant-system/kuadrant-operator-catalog-2qw4q"
Jul 15 12:16:04.216868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.216828    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q"
Jul 15 12:16:04.358938 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.358904    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2qw4q"]
Jul 15 12:16:04.372420 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.372395    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2qw4q"]
Jul 15 12:16:04.374939 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:04.374909    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62e2de90_103b_4c1e_8036_4de62451c46e.slice/crio-e14760dbde09d0d5bf65125a921e7d4560f007f7a1b7cc486b46adef8b958d09 WatchSource:0}: Error finding container e14760dbde09d0d5bf65125a921e7d4560f007f7a1b7cc486b46adef8b958d09: Status 404 returned error can't find the container with id e14760dbde09d0d5bf65125a921e7d4560f007f7a1b7cc486b46adef8b958d09
Jul 15 12:16:04.482109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.482072    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6vfqz"]
Jul 15 12:16:04.485003 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.484980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:04.499461 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.499429    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6vfqz"]
Jul 15 12:16:04.642285 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.642244    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7scz\" (UniqueName: \"kubernetes.io/projected/216d8708-ab10-494c-ba44-e0f7d30e0875-kube-api-access-r7scz\") pod \"kuadrant-operator-catalog-6vfqz\" (UID: \"216d8708-ab10-494c-ba44-e0f7d30e0875\") " pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:04.743464 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.743374    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7scz\" (UniqueName: \"kubernetes.io/projected/216d8708-ab10-494c-ba44-e0f7d30e0875-kube-api-access-r7scz\") pod \"kuadrant-operator-catalog-6vfqz\" (UID: \"216d8708-ab10-494c-ba44-e0f7d30e0875\") " pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:04.751936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.751901    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7scz\" (UniqueName: \"kubernetes.io/projected/216d8708-ab10-494c-ba44-e0f7d30e0875-kube-api-access-r7scz\") pod \"kuadrant-operator-catalog-6vfqz\" (UID: \"216d8708-ab10-494c-ba44-e0f7d30e0875\") " pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:04.794171 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.794136    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:04.938480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:04.938449    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6vfqz"]
Jul 15 12:16:04.940276 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:04.940248    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216d8708_ab10_494c_ba44_e0f7d30e0875.slice/crio-790234f8d4d04d61c7c555e0f63b64faceebb332540aa69b2ade2cfd2f3d8df6 WatchSource:0}: Error finding container 790234f8d4d04d61c7c555e0f63b64faceebb332540aa69b2ade2cfd2f3d8df6: Status 404 returned error can't find the container with id 790234f8d4d04d61c7c555e0f63b64faceebb332540aa69b2ade2cfd2f3d8df6
Jul 15 12:16:05.036115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:05.036004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz" event={"ID":"216d8708-ab10-494c-ba44-e0f7d30e0875","Type":"ContainerStarted","Data":"790234f8d4d04d61c7c555e0f63b64faceebb332540aa69b2ade2cfd2f3d8df6"}
Jul 15 12:16:05.037342 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:05.037316    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q" event={"ID":"62e2de90-103b-4c1e-8036-4de62451c46e","Type":"ContainerStarted","Data":"e14760dbde09d0d5bf65125a921e7d4560f007f7a1b7cc486b46adef8b958d09"}
Jul 15 12:16:07.049173 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.049135    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz" event={"ID":"216d8708-ab10-494c-ba44-e0f7d30e0875","Type":"ContainerStarted","Data":"cdf815d785c5d18c54c0099f7325ff75b590befc72a01581eede5c809c3233b6"}
Jul 15 12:16:07.050563 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.050539    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q" event={"ID":"62e2de90-103b-4c1e-8036-4de62451c46e","Type":"ContainerStarted","Data":"b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506"}
Jul 15 12:16:07.050691 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.050588    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q" podUID="62e2de90-103b-4c1e-8036-4de62451c46e" containerName="registry-server" containerID="cri-o://b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506" gracePeriod=2
Jul 15 12:16:07.071128 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.071064    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz" podStartSLOduration=1.5404114880000002 podStartE2EDuration="3.071048824s" totalImagesPullingTime="1.530637336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:16:07.069527125 +0000 UTC m=+498.169476466" watchObservedRunningTime="2026-07-15 12:16:07.071048824 +0000 UTC m=+498.170998141"
Jul 15 12:16:07.092861 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.092700    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q" podStartSLOduration=2.006759413 podStartE2EDuration="4.092677764s" totalImagesPullingTime="2.085918351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:16:07.089376286 +0000 UTC m=+498.189325756" watchObservedRunningTime="2026-07-15 12:16:07.092677764 +0000 UTC m=+498.192627104"
Jul 15 12:16:07.286866 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.286838    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q"
Jul 15 12:16:07.370611 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.370524    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mq5n\" (UniqueName: \"kubernetes.io/projected/62e2de90-103b-4c1e-8036-4de62451c46e-kube-api-access-4mq5n\") pod \"62e2de90-103b-4c1e-8036-4de62451c46e\" (UID: \"62e2de90-103b-4c1e-8036-4de62451c46e\") "
Jul 15 12:16:07.372637 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.372613    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62e2de90-103b-4c1e-8036-4de62451c46e-kube-api-access-4mq5n" (OuterVolumeSpecName: "kube-api-access-4mq5n") pod "62e2de90-103b-4c1e-8036-4de62451c46e" (UID: "62e2de90-103b-4c1e-8036-4de62451c46e"). InnerVolumeSpecName "kube-api-access-4mq5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:16:07.471278 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:07.471240    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mq5n\" (UniqueName: \"kubernetes.io/projected/62e2de90-103b-4c1e-8036-4de62451c46e-kube-api-access-4mq5n\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:08.058397 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.058366    2582 generic.go:356] "Generic (PLEG): container finished" podID="62e2de90-103b-4c1e-8036-4de62451c46e" containerID="b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506" exitCode=0
Jul 15 12:16:08.058853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.058588    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q"
Jul 15 12:16:08.061645 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.061616    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q" event={"ID":"62e2de90-103b-4c1e-8036-4de62451c46e","Type":"ContainerDied","Data":"b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506"}
Jul 15 12:16:08.061798 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.061655    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2qw4q" event={"ID":"62e2de90-103b-4c1e-8036-4de62451c46e","Type":"ContainerDied","Data":"e14760dbde09d0d5bf65125a921e7d4560f007f7a1b7cc486b46adef8b958d09"}
Jul 15 12:16:08.061798 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.061677    2582 scope.go:117] "RemoveContainer" containerID="b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506"
Jul 15 12:16:08.070855 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.070836    2582 scope.go:117] "RemoveContainer" containerID="b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506"
Jul 15 12:16:08.071118 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:08.071075    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506\": container with ID starting with b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506 not found: ID does not exist" containerID="b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506"
Jul 15 12:16:08.071196 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.071126    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506"} err="failed to get container status \"b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506\": rpc error: code = NotFound desc = could not find container \"b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506\": container with ID starting with b36e98d08b33c6c83fb3b60b586234d698397abef4d0225af7974ac15ac8b506 not found: ID does not exist"
Jul 15 12:16:08.084311 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.084284    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2qw4q"]
Jul 15 12:16:08.091793 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:08.091771    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2qw4q"]
Jul 15 12:16:10.062257 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:10.062226    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62e2de90-103b-4c1e-8036-4de62451c46e" path="/var/lib/kubelet/pods/62e2de90-103b-4c1e-8036-4de62451c46e/volumes"
Jul 15 12:16:14.794594 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:14.794552    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:14.795018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:14.794620    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:14.817244 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:14.817213    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:15.116161 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:15.116078    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6vfqz"
Jul 15 12:16:19.520851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.520814    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"]
Jul 15 12:16:19.521247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.521167    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e2de90-103b-4c1e-8036-4de62451c46e" containerName="registry-server"
Jul 15 12:16:19.521247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.521179    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e2de90-103b-4c1e-8036-4de62451c46e" containerName="registry-server"
Jul 15 12:16:19.521247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.521242    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e2de90-103b-4c1e-8036-4de62451c46e" containerName="registry-server"
Jul 15 12:16:19.524566 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.524538    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.527189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.527166    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d55pm\""
Jul 15 12:16:19.536373 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.536349    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"]
Jul 15 12:16:19.672512 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.672466    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.672728 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.672531    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.672728 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.672628    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl2pq\" (UniqueName: \"kubernetes.io/projected/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-kube-api-access-nl2pq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.773969 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.773865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.773969 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.773938    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.774219 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.773982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl2pq\" (UniqueName: \"kubernetes.io/projected/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-kube-api-access-nl2pq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.774366 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.774345    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.774433 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.774380    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.783135 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.783117    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl2pq\" (UniqueName: \"kubernetes.io/projected/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-kube-api-access-nl2pq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.833176 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.833143    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:19.923475 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.923438    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"]
Jul 15 12:16:19.927825 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.927801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:19.935300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.935233    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"]
Jul 15 12:16:19.967682 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:19.967653    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"]
Jul 15 12:16:19.969994 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:19.969967    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0ca6bc_8dd3_477e_ba10_3eeed095742d.slice/crio-75286944bbb2a00052430095f1fb830081376de616884d293179bb29007d6153 WatchSource:0}: Error finding container 75286944bbb2a00052430095f1fb830081376de616884d293179bb29007d6153: Status 404 returned error can't find the container with id 75286944bbb2a00052430095f1fb830081376de616884d293179bb29007d6153
Jul 15 12:16:20.075405 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.075375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.075515 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.075427    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.075515 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.075482    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2whj8\" (UniqueName: \"kubernetes.io/projected/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-kube-api-access-2whj8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.117815 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.117780    2582 generic.go:356] "Generic (PLEG): container finished" podID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerID="0059962d9f5015452073e6fbe50e5fc44f20cf658759cfe5137e0308e5e1238b" exitCode=0
Jul 15 12:16:20.117988 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.117868    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2" event={"ID":"fb0ca6bc-8dd3-477e-ba10-3eeed095742d","Type":"ContainerDied","Data":"0059962d9f5015452073e6fbe50e5fc44f20cf658759cfe5137e0308e5e1238b"}
Jul 15 12:16:20.117988 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.117921    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2" event={"ID":"fb0ca6bc-8dd3-477e-ba10-3eeed095742d","Type":"ContainerStarted","Data":"75286944bbb2a00052430095f1fb830081376de616884d293179bb29007d6153"}
Jul 15 12:16:20.176439 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.176360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2whj8\" (UniqueName: \"kubernetes.io/projected/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-kube-api-access-2whj8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.176646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.176470    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.176646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.176536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.176907 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.176881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.176977 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.176892    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.185961 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.185930    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2whj8\" (UniqueName: \"kubernetes.io/projected/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-kube-api-access-2whj8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.240923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.240889    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:20.312310 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.312277    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"]
Jul 15 12:16:20.316283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.316215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.323623 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.323586    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"]
Jul 15 12:16:20.382078 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.382053    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"]
Jul 15 12:16:20.384028 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:20.384002    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b67f40_3daf_4e0f_a45f_c4cff2c91d9b.slice/crio-14fd02a67c1499c8d9142f098c8d122d1ed70a23e1ee21c25a3d16719fe3878f WatchSource:0}: Error finding container 14fd02a67c1499c8d9142f098c8d122d1ed70a23e1ee21c25a3d16719fe3878f: Status 404 returned error can't find the container with id 14fd02a67c1499c8d9142f098c8d122d1ed70a23e1ee21c25a3d16719fe3878f
Jul 15 12:16:20.480067 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.480027    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh8dl\" (UniqueName: \"kubernetes.io/projected/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-kube-api-access-fh8dl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.480067 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.480070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.480317 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.480184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.581341 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.581240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh8dl\" (UniqueName: \"kubernetes.io/projected/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-kube-api-access-fh8dl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.581341 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.581291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.581864 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.581370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.581864 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.581757    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.581864 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.581812    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.590878 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.590851    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh8dl\" (UniqueName: \"kubernetes.io/projected/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-kube-api-access-fh8dl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.627219 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.627184    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:20.716060 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.716026    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"]
Jul 15 12:16:20.721316 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.721290    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.728645 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.728616    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"]
Jul 15 12:16:20.774997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.772731    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"]
Jul 15 12:16:20.774997 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:20.772839    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573c3e52_822b_4b48_b2f6_dd3f198f9b8b.slice/crio-a85aec5299a128903a35a40ec8397c2a1d00e40951ca63b373fbad771cce42aa WatchSource:0}: Error finding container a85aec5299a128903a35a40ec8397c2a1d00e40951ca63b373fbad771cce42aa: Status 404 returned error can't find the container with id a85aec5299a128903a35a40ec8397c2a1d00e40951ca63b373fbad771cce42aa
Jul 15 12:16:20.883674 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.883642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-587lf\" (UniqueName: \"kubernetes.io/projected/db0aa6ab-0337-44c5-a70e-0340151cf8e2-kube-api-access-587lf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.883829 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.883694    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.883829 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.883734    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.984202 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.984171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-587lf\" (UniqueName: \"kubernetes.io/projected/db0aa6ab-0337-44c5-a70e-0340151cf8e2-kube-api-access-587lf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.984372 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.984208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.984372 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.984227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.984591 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.984576    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.984637 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.984574    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:20.998741 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:20.998713    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-587lf\" (UniqueName: \"kubernetes.io/projected/db0aa6ab-0337-44c5-a70e-0340151cf8e2-kube-api-access-587lf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:21.032557 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.032514    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:21.125723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.125316    2582 generic.go:356] "Generic (PLEG): container finished" podID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerID="614ca76827e9fc4db1f5dd0459c2cb442481571269f0ecbffe95c5ce8549bbf3" exitCode=0
Jul 15 12:16:21.125723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.125404    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt" event={"ID":"573c3e52-822b-4b48-b2f6-dd3f198f9b8b","Type":"ContainerDied","Data":"614ca76827e9fc4db1f5dd0459c2cb442481571269f0ecbffe95c5ce8549bbf3"}
Jul 15 12:16:21.125723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.125470    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt" event={"ID":"573c3e52-822b-4b48-b2f6-dd3f198f9b8b","Type":"ContainerStarted","Data":"a85aec5299a128903a35a40ec8397c2a1d00e40951ca63b373fbad771cce42aa"}
Jul 15 12:16:21.127998 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.127432    2582 generic.go:356] "Generic (PLEG): container finished" podID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerID="3184740fffc5356826d9f85651358a2156683822d14104e90e235f746846b16d" exitCode=0
Jul 15 12:16:21.127998 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.127523    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59" event={"ID":"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b","Type":"ContainerDied","Data":"3184740fffc5356826d9f85651358a2156683822d14104e90e235f746846b16d"}
Jul 15 12:16:21.127998 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.127564    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59" event={"ID":"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b","Type":"ContainerStarted","Data":"14fd02a67c1499c8d9142f098c8d122d1ed70a23e1ee21c25a3d16719fe3878f"}
Jul 15 12:16:21.132690 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.132660    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2" event={"ID":"fb0ca6bc-8dd3-477e-ba10-3eeed095742d","Type":"ContainerStarted","Data":"5952f0c75747d8afee3fcb53b8a7bb416c8809a0f09b1b02a0f8f7543326d999"}
Jul 15 12:16:21.187822 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:21.187800    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"]
Jul 15 12:16:21.193001 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:21.192977    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0aa6ab_0337_44c5_a70e_0340151cf8e2.slice/crio-80f797bf093bc9def075d72b91d9da20ef5593125b430eb4e59c9fc41a9e0ab2 WatchSource:0}: Error finding container 80f797bf093bc9def075d72b91d9da20ef5593125b430eb4e59c9fc41a9e0ab2: Status 404 returned error can't find the container with id 80f797bf093bc9def075d72b91d9da20ef5593125b430eb4e59c9fc41a9e0ab2
Jul 15 12:16:22.140745 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:22.140703    2582 generic.go:356] "Generic (PLEG): container finished" podID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerID="5952f0c75747d8afee3fcb53b8a7bb416c8809a0f09b1b02a0f8f7543326d999" exitCode=0
Jul 15 12:16:22.141114 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:22.140783    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2" event={"ID":"fb0ca6bc-8dd3-477e-ba10-3eeed095742d","Type":"ContainerDied","Data":"5952f0c75747d8afee3fcb53b8a7bb416c8809a0f09b1b02a0f8f7543326d999"}
Jul 15 12:16:22.142703 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:22.142678    2582 generic.go:356] "Generic (PLEG): container finished" podID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerID="56f3f63e325c29c444d243a318533dc63fde0c106c804f22fba98f520145bd70" exitCode=0
Jul 15 12:16:22.142814 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:22.142704    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz" event={"ID":"db0aa6ab-0337-44c5-a70e-0340151cf8e2","Type":"ContainerDied","Data":"56f3f63e325c29c444d243a318533dc63fde0c106c804f22fba98f520145bd70"}
Jul 15 12:16:22.142814 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:22.142749    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz" event={"ID":"db0aa6ab-0337-44c5-a70e-0340151cf8e2","Type":"ContainerStarted","Data":"80f797bf093bc9def075d72b91d9da20ef5593125b430eb4e59c9fc41a9e0ab2"}
Jul 15 12:16:23.149970 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.149934    2582 generic.go:356] "Generic (PLEG): container finished" podID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerID="417827d29725f75cdecdab0012ccc212a5a1bcbf40af6dc96a843a87c24c893d" exitCode=0
Jul 15 12:16:23.150356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.150013    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2" event={"ID":"fb0ca6bc-8dd3-477e-ba10-3eeed095742d","Type":"ContainerDied","Data":"417827d29725f75cdecdab0012ccc212a5a1bcbf40af6dc96a843a87c24c893d"}
Jul 15 12:16:23.151681 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.151659    2582 generic.go:356] "Generic (PLEG): container finished" podID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerID="e63d30022a7de29bf95dd4667412bd5e622ea7541b029fef9b974101d47bdb1c" exitCode=0
Jul 15 12:16:23.151748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.151731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz" event={"ID":"db0aa6ab-0337-44c5-a70e-0340151cf8e2","Type":"ContainerDied","Data":"e63d30022a7de29bf95dd4667412bd5e622ea7541b029fef9b974101d47bdb1c"}
Jul 15 12:16:23.153601 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.153581    2582 generic.go:356] "Generic (PLEG): container finished" podID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerID="544179c47eb2c138331dd0c6bd568a99a1b1b08659f03f7ba8847f1eeab63ca3" exitCode=0
Jul 15 12:16:23.153707 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.153672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt" event={"ID":"573c3e52-822b-4b48-b2f6-dd3f198f9b8b","Type":"ContainerDied","Data":"544179c47eb2c138331dd0c6bd568a99a1b1b08659f03f7ba8847f1eeab63ca3"}
Jul 15 12:16:23.155451 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.155428    2582 generic.go:356] "Generic (PLEG): container finished" podID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerID="b558a017fede1979cb4475674951604ffe5e4e6321a7da77ebfab72f50fc70d0" exitCode=0
Jul 15 12:16:23.155531 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:23.155460    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59" event={"ID":"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b","Type":"ContainerDied","Data":"b558a017fede1979cb4475674951604ffe5e4e6321a7da77ebfab72f50fc70d0"}
Jul 15 12:16:24.162647 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.162553    2582 generic.go:356] "Generic (PLEG): container finished" podID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerID="c8d14f8083d66b80cbb17afc170b86a1673e7d2f42883da0bac7273ed2adf6a9" exitCode=0
Jul 15 12:16:24.162979 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.162642    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt" event={"ID":"573c3e52-822b-4b48-b2f6-dd3f198f9b8b","Type":"ContainerDied","Data":"c8d14f8083d66b80cbb17afc170b86a1673e7d2f42883da0bac7273ed2adf6a9"}
Jul 15 12:16:24.164661 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.164635    2582 generic.go:356] "Generic (PLEG): container finished" podID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerID="e5da61f4f118adc476b25537e82e36bfe8bd67d91750d4d89e31d548e5ea8c1d" exitCode=0
Jul 15 12:16:24.164774 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.164711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59" event={"ID":"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b","Type":"ContainerDied","Data":"e5da61f4f118adc476b25537e82e36bfe8bd67d91750d4d89e31d548e5ea8c1d"}
Jul 15 12:16:24.166888 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.166866    2582 generic.go:356] "Generic (PLEG): container finished" podID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerID="8302f4774b23eeef76e8d8834da4874f95c5c3e8a49bd9885cd6d9dfc935420e" exitCode=0
Jul 15 12:16:24.167027 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.166922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz" event={"ID":"db0aa6ab-0337-44c5-a70e-0340151cf8e2","Type":"ContainerDied","Data":"8302f4774b23eeef76e8d8834da4874f95c5c3e8a49bd9885cd6d9dfc935420e"}
Jul 15 12:16:24.301344 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.301319    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:24.311908 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.311877    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-util\") pod \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") "
Jul 15 12:16:24.312016 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.311920    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-bundle\") pod \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") "
Jul 15 12:16:24.312016 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.312000    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl2pq\" (UniqueName: \"kubernetes.io/projected/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-kube-api-access-nl2pq\") pod \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\" (UID: \"fb0ca6bc-8dd3-477e-ba10-3eeed095742d\") "
Jul 15 12:16:24.312567 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.312536    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-bundle" (OuterVolumeSpecName: "bundle") pod "fb0ca6bc-8dd3-477e-ba10-3eeed095742d" (UID: "fb0ca6bc-8dd3-477e-ba10-3eeed095742d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:24.314141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.314106    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-kube-api-access-nl2pq" (OuterVolumeSpecName: "kube-api-access-nl2pq") pod "fb0ca6bc-8dd3-477e-ba10-3eeed095742d" (UID: "fb0ca6bc-8dd3-477e-ba10-3eeed095742d"). InnerVolumeSpecName "kube-api-access-nl2pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:16:24.317742 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.317716    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-util" (OuterVolumeSpecName: "util") pod "fb0ca6bc-8dd3-477e-ba10-3eeed095742d" (UID: "fb0ca6bc-8dd3-477e-ba10-3eeed095742d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:24.412814 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.412732    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl2pq\" (UniqueName: \"kubernetes.io/projected/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-kube-api-access-nl2pq\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:24.412814 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.412761    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-util\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:24.412814 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:24.412771    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb0ca6bc-8dd3-477e-ba10-3eeed095742d-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.173784 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.173753    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2"
Jul 15 12:16:25.173784 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.173780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2" event={"ID":"fb0ca6bc-8dd3-477e-ba10-3eeed095742d","Type":"ContainerDied","Data":"75286944bbb2a00052430095f1fb830081376de616884d293179bb29007d6153"}
Jul 15 12:16:25.174315 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.173837    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75286944bbb2a00052430095f1fb830081376de616884d293179bb29007d6153"
Jul 15 12:16:25.323713 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.323689    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:25.361343 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.361304    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:25.364459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.364437    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:25.418795 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.418761    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-util\") pod \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") "
Jul 15 12:16:25.419028 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.418814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2whj8\" (UniqueName: \"kubernetes.io/projected/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-kube-api-access-2whj8\") pod \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") "
Jul 15 12:16:25.419028 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.418844    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-bundle\") pod \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") "
Jul 15 12:16:25.419028 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.418867    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh8dl\" (UniqueName: \"kubernetes.io/projected/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-kube-api-access-fh8dl\") pod \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") "
Jul 15 12:16:25.419879 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.419539    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-bundle" (OuterVolumeSpecName: "bundle") pod "65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" (UID: "65b67f40-3daf-4e0f-a45f-c4cff2c91d9b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:25.422377 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.422319    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-kube-api-access-2whj8" (OuterVolumeSpecName: "kube-api-access-2whj8") pod "65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" (UID: "65b67f40-3daf-4e0f-a45f-c4cff2c91d9b"). InnerVolumeSpecName "kube-api-access-2whj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:16:25.423160 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.423130    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-kube-api-access-fh8dl" (OuterVolumeSpecName: "kube-api-access-fh8dl") pod "573c3e52-822b-4b48-b2f6-dd3f198f9b8b" (UID: "573c3e52-822b-4b48-b2f6-dd3f198f9b8b"). InnerVolumeSpecName "kube-api-access-fh8dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:16:25.426928 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.426862    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-util" (OuterVolumeSpecName: "util") pod "573c3e52-822b-4b48-b2f6-dd3f198f9b8b" (UID: "573c3e52-822b-4b48-b2f6-dd3f198f9b8b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:25.519973 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.519918    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-bundle\") pod \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") "
Jul 15 12:16:25.519973 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.519974    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-util\") pod \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") "
Jul 15 12:16:25.520269 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.519995    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-util\") pod \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\" (UID: \"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b\") "
Jul 15 12:16:25.520269 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520013    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-bundle\") pod \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\" (UID: \"573c3e52-822b-4b48-b2f6-dd3f198f9b8b\") "
Jul 15 12:16:25.520269 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520039    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-587lf\" (UniqueName: \"kubernetes.io/projected/db0aa6ab-0337-44c5-a70e-0340151cf8e2-kube-api-access-587lf\") pod \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\" (UID: \"db0aa6ab-0337-44c5-a70e-0340151cf8e2\") "
Jul 15 12:16:25.520269 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520255    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-util\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.520269 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520266    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2whj8\" (UniqueName: \"kubernetes.io/projected/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-kube-api-access-2whj8\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.520523 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520275    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.520523 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520283    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh8dl\" (UniqueName: \"kubernetes.io/projected/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-kube-api-access-fh8dl\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.520523 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520490    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-bundle" (OuterVolumeSpecName: "bundle") pod "db0aa6ab-0337-44c5-a70e-0340151cf8e2" (UID: "db0aa6ab-0337-44c5-a70e-0340151cf8e2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:25.520711 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.520691    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-bundle" (OuterVolumeSpecName: "bundle") pod "573c3e52-822b-4b48-b2f6-dd3f198f9b8b" (UID: "573c3e52-822b-4b48-b2f6-dd3f198f9b8b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:25.522376 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.522349    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db0aa6ab-0337-44c5-a70e-0340151cf8e2-kube-api-access-587lf" (OuterVolumeSpecName: "kube-api-access-587lf") pod "db0aa6ab-0337-44c5-a70e-0340151cf8e2" (UID: "db0aa6ab-0337-44c5-a70e-0340151cf8e2"). InnerVolumeSpecName "kube-api-access-587lf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:16:25.528375 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.528328    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-util" (OuterVolumeSpecName: "util") pod "65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" (UID: "65b67f40-3daf-4e0f-a45f-c4cff2c91d9b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:25.529002 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.528982    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-util" (OuterVolumeSpecName: "util") pod "db0aa6ab-0337-44c5-a70e-0340151cf8e2" (UID: "db0aa6ab-0337-44c5-a70e-0340151cf8e2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:16:25.620825 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.620772    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573c3e52-822b-4b48-b2f6-dd3f198f9b8b-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.620825 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.620819    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-587lf\" (UniqueName: \"kubernetes.io/projected/db0aa6ab-0337-44c5-a70e-0340151cf8e2-kube-api-access-587lf\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.620825 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.620831    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.620825 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.620839    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db0aa6ab-0337-44c5-a70e-0340151cf8e2-util\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:25.621086 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:25.620848    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65b67f40-3daf-4e0f-a45f-c4cff2c91d9b-util\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:16:26.180951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.180918    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt"
Jul 15 12:16:26.180951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.180928    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt" event={"ID":"573c3e52-822b-4b48-b2f6-dd3f198f9b8b","Type":"ContainerDied","Data":"a85aec5299a128903a35a40ec8397c2a1d00e40951ca63b373fbad771cce42aa"}
Jul 15 12:16:26.181488 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.181008    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a85aec5299a128903a35a40ec8397c2a1d00e40951ca63b373fbad771cce42aa"
Jul 15 12:16:26.183008 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.182984    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59"
Jul 15 12:16:26.183170 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.182984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59" event={"ID":"65b67f40-3daf-4e0f-a45f-c4cff2c91d9b","Type":"ContainerDied","Data":"14fd02a67c1499c8d9142f098c8d122d1ed70a23e1ee21c25a3d16719fe3878f"}
Jul 15 12:16:26.183245 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.183192    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14fd02a67c1499c8d9142f098c8d122d1ed70a23e1ee21c25a3d16719fe3878f"
Jul 15 12:16:26.185403 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.185376    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz" event={"ID":"db0aa6ab-0337-44c5-a70e-0340151cf8e2","Type":"ContainerDied","Data":"80f797bf093bc9def075d72b91d9da20ef5593125b430eb4e59c9fc41a9e0ab2"}
Jul 15 12:16:26.185403 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.185394    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz"
Jul 15 12:16:26.185549 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:26.185415    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80f797bf093bc9def075d72b91d9da20ef5593125b430eb4e59c9fc41a9e0ab2"
Jul 15 12:16:37.121950 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.121905    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"]
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122225    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122236    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122246    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122251    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122257    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122263    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122269    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122274    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122279    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122284    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122291    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122296    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122309    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122313    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122318    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122324    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122330    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122335    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerName="pull"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122341    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122347    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerName="util"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122353    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122357    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122362    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122367    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122423    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb0ca6bc-8dd3-477e-ba10-3eeed095742d" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122430    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b67f40-3daf-4e0f-a45f-c4cff2c91d9b" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122437    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0aa6ab-0337-44c5-a70e-0340151cf8e2" containerName="extract"
Jul 15 12:16:37.122459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.122443    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="573c3e52-822b-4b48-b2f6-dd3f198f9b8b" containerName="extract"
Jul 15 12:16:37.126813 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.126791    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.129024 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.128999    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vjlzw\""
Jul 15 12:16:37.137466 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.137430    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"]
Jul 15 12:16:37.207351 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.207306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77pm\" (UniqueName: \"kubernetes.io/projected/b8d29787-cce7-447b-9f21-8abd816ed7da-kube-api-access-s77pm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.207351 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.207354    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8d29787-cce7-447b-9f21-8abd816ed7da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.308473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.308414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77pm\" (UniqueName: \"kubernetes.io/projected/b8d29787-cce7-447b-9f21-8abd816ed7da-kube-api-access-s77pm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.308473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.308475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8d29787-cce7-447b-9f21-8abd816ed7da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.308825 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.308805    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8d29787-cce7-447b-9f21-8abd816ed7da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.324158 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.324122    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77pm\" (UniqueName: \"kubernetes.io/projected/b8d29787-cce7-447b-9f21-8abd816ed7da-kube-api-access-s77pm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.439167 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.439061    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:37.584280 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:37.584253    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"]
Jul 15 12:16:37.586872 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:37.586841    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d29787_cce7_447b_9f21_8abd816ed7da.slice/crio-18dda51686ca0e009ba8fa086f45205ed80ead5c67397696fae9dd5a86043613 WatchSource:0}: Error finding container 18dda51686ca0e009ba8fa086f45205ed80ead5c67397696fae9dd5a86043613: Status 404 returned error can't find the container with id 18dda51686ca0e009ba8fa086f45205ed80ead5c67397696fae9dd5a86043613
Jul 15 12:16:38.249974 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:38.249936    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x" event={"ID":"b8d29787-cce7-447b-9f21-8abd816ed7da","Type":"ContainerStarted","Data":"18dda51686ca0e009ba8fa086f45205ed80ead5c67397696fae9dd5a86043613"}
Jul 15 12:16:39.172662 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.172625    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"]
Jul 15 12:16:39.175175 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.175153    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:39.180044 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.180020    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bfklk\""
Jul 15 12:16:39.216234 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.216198    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"]
Jul 15 12:16:39.226556 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.226523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ztqp\" (UniqueName: \"kubernetes.io/projected/d9d69db3-aab7-43c1-b5b1-675b353ec965-kube-api-access-6ztqp\") pod \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" (UID: \"d9d69db3-aab7-43c1-b5b1-675b353ec965\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:39.328119 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.328052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztqp\" (UniqueName: \"kubernetes.io/projected/d9d69db3-aab7-43c1-b5b1-675b353ec965-kube-api-access-6ztqp\") pod \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" (UID: \"d9d69db3-aab7-43c1-b5b1-675b353ec965\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:39.343510 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.343478    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztqp\" (UniqueName: \"kubernetes.io/projected/d9d69db3-aab7-43c1-b5b1-675b353ec965-kube-api-access-6ztqp\") pod \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" (UID: \"d9d69db3-aab7-43c1-b5b1-675b353ec965\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:39.485458 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.485421    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:39.644716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:39.644686    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"]
Jul 15 12:16:39.646408 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:39.646375    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d69db3_aab7_43c1_b5b1_675b353ec965.slice/crio-cab99acb4e60996934f292741310c773f28cfc24f05cb72c26882acf51b78942 WatchSource:0}: Error finding container cab99acb4e60996934f292741310c773f28cfc24f05cb72c26882acf51b78942: Status 404 returned error can't find the container with id cab99acb4e60996934f292741310c773f28cfc24f05cb72c26882acf51b78942
Jul 15 12:16:40.262414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:40.262383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl" event={"ID":"d9d69db3-aab7-43c1-b5b1-675b353ec965","Type":"ContainerStarted","Data":"cab99acb4e60996934f292741310c773f28cfc24f05cb72c26882acf51b78942"}
Jul 15 12:16:42.281522 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:42.281390    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl" event={"ID":"d9d69db3-aab7-43c1-b5b1-675b353ec965","Type":"ContainerStarted","Data":"2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86"}
Jul 15 12:16:42.281522 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:42.281505    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:42.318352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:42.318288    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl" podStartSLOduration=0.955466179 podStartE2EDuration="3.318271491s" totalImagesPullingTime="2.362805312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:16:42.312227151 +0000 UTC m=+533.412176494" watchObservedRunningTime="2026-07-15 12:16:42.318271491 +0000 UTC m=+533.418220829"
Jul 15 12:16:45.033258 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.033167    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"]
Jul 15 12:16:45.037271 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.037249    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"
Jul 15 12:16:45.040743 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.040717    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 12:16:45.040849 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.040767    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pdqxb\""
Jul 15 12:16:45.049028 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.049004    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"]
Jul 15 12:16:45.080598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.080568    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjsrf\" (UniqueName: \"kubernetes.io/projected/a6ac67d2-d0d9-4917-b077-5f8ab516e89d-kube-api-access-fjsrf\") pod \"dns-operator-controller-manager-864dd78c4c-rdr4n\" (UID: \"a6ac67d2-d0d9-4917-b077-5f8ab516e89d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"
Jul 15 12:16:45.182015 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.181976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjsrf\" (UniqueName: \"kubernetes.io/projected/a6ac67d2-d0d9-4917-b077-5f8ab516e89d-kube-api-access-fjsrf\") pod \"dns-operator-controller-manager-864dd78c4c-rdr4n\" (UID: \"a6ac67d2-d0d9-4917-b077-5f8ab516e89d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"
Jul 15 12:16:45.203644 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.203608    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjsrf\" (UniqueName: \"kubernetes.io/projected/a6ac67d2-d0d9-4917-b077-5f8ab516e89d-kube-api-access-fjsrf\") pod \"dns-operator-controller-manager-864dd78c4c-rdr4n\" (UID: \"a6ac67d2-d0d9-4917-b077-5f8ab516e89d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"
Jul 15 12:16:45.297470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.297381    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x" event={"ID":"b8d29787-cce7-447b-9f21-8abd816ed7da","Type":"ContainerStarted","Data":"ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75"}
Jul 15 12:16:45.297470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.297460    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:45.319369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.319319    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x" podStartSLOduration=1.5966596690000001 podStartE2EDuration="8.319302709s" totalImagesPullingTime="6.72264304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:16:45.31827564 +0000 UTC m=+536.418225013" watchObservedRunningTime="2026-07-15 12:16:45.319302709 +0000 UTC m=+536.419252048"
Jul 15 12:16:45.348272 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.348236    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"
Jul 15 12:16:45.491218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:45.491194    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"]
Jul 15 12:16:45.493125 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:45.493073    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ac67d2_d0d9_4917_b077_5f8ab516e89d.slice/crio-d505dc102f1602422f29c9221a85c5dd9ec67450af5c3a1df8c7a6b7e19f0e36 WatchSource:0}: Error finding container d505dc102f1602422f29c9221a85c5dd9ec67450af5c3a1df8c7a6b7e19f0e36: Status 404 returned error can't find the container with id d505dc102f1602422f29c9221a85c5dd9ec67450af5c3a1df8c7a6b7e19f0e36
Jul 15 12:16:46.304957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:46.304923    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n" event={"ID":"a6ac67d2-d0d9-4917-b077-5f8ab516e89d","Type":"ContainerStarted","Data":"d505dc102f1602422f29c9221a85c5dd9ec67450af5c3a1df8c7a6b7e19f0e36"}
Jul 15 12:16:47.363689 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.363652    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"]
Jul 15 12:16:47.367014 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.366992    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.370352 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.370333    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d55pm\""
Jul 15 12:16:47.370444 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.370395    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 15 12:16:47.370989 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.370974    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 15 12:16:47.400424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.400393    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"]
Jul 15 12:16:47.403055 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.403030    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/598cede8-3607-497d-b472-43800fa6010e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.403255 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.403074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mhpz\" (UniqueName: \"kubernetes.io/projected/598cede8-3607-497d-b472-43800fa6010e-kube-api-access-7mhpz\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.403255 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.403178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/598cede8-3607-497d-b472-43800fa6010e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.504002 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.503969    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/598cede8-3607-497d-b472-43800fa6010e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.504002 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.504017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mhpz\" (UniqueName: \"kubernetes.io/projected/598cede8-3607-497d-b472-43800fa6010e-kube-api-access-7mhpz\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.504266 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.504072    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/598cede8-3607-497d-b472-43800fa6010e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.504889 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.504863    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/598cede8-3607-497d-b472-43800fa6010e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.506824 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.506797    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/598cede8-3607-497d-b472-43800fa6010e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.517239 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.517212    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mhpz\" (UniqueName: \"kubernetes.io/projected/598cede8-3607-497d-b472-43800fa6010e-kube-api-access-7mhpz\") pod \"kuadrant-console-plugin-66b5774985-s2n87\" (UID: \"598cede8-3607-497d-b472-43800fa6010e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.676309 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.676208    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"
Jul 15 12:16:47.882153 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:47.882129    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87"]
Jul 15 12:16:47.884361 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:16:47.884335    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod598cede8_3607_497d_b472_43800fa6010e.slice/crio-58b5ee9e4a0bcf726376206ab78ff53b363e132810b054e6b238b74458508981 WatchSource:0}: Error finding container 58b5ee9e4a0bcf726376206ab78ff53b363e132810b054e6b238b74458508981: Status 404 returned error can't find the container with id 58b5ee9e4a0bcf726376206ab78ff53b363e132810b054e6b238b74458508981
Jul 15 12:16:48.317616 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:48.317577    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n" event={"ID":"a6ac67d2-d0d9-4917-b077-5f8ab516e89d","Type":"ContainerStarted","Data":"ad7cee30d1c34f2005cffb834dcd3b1e9f888f7b7c5d32c75829cf73ab5fa4c8"}
Jul 15 12:16:48.317845 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:48.317681    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"
Jul 15 12:16:48.318853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:48.318832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87" event={"ID":"598cede8-3607-497d-b472-43800fa6010e","Type":"ContainerStarted","Data":"58b5ee9e4a0bcf726376206ab78ff53b363e132810b054e6b238b74458508981"}
Jul 15 12:16:48.345949 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:48.345897    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n" podStartSLOduration=1.063764522 podStartE2EDuration="3.345868377s" totalImagesPullingTime="2.282103855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:16:48.3450811 +0000 UTC m=+539.445030438" watchObservedRunningTime="2026-07-15 12:16:48.345868377 +0000 UTC m=+539.445817715"
Jul 15 12:16:53.290531 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:53.290482    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:56.307345 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:56.307312    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:58.342775 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.342744    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"]
Jul 15 12:16:58.343231 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.342966    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" containerName="manager" containerID="cri-o://ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75" gracePeriod=2
Jul 15 12:16:58.354723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.354692    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"]
Jul 15 12:16:58.369111 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.369055    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"]
Jul 15 12:16:58.369331 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.369305    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" containerName="manager" containerID="cri-o://2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86" gracePeriod=2
Jul 15 12:16:58.381172 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.381000    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"]
Jul 15 12:16:58.381853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.381746    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" containerName="manager"
Jul 15 12:16:58.381853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.381770    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" containerName="manager"
Jul 15 12:16:58.382274 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.382146    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" containerName="manager"
Jul 15 12:16:58.383288 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:58.383225    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:58.385660 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.385628    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"]
Jul 15 12:16:58.385792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.385768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.388113 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:58.388047    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:58.397142 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.397108    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"]
Jul 15 12:16:58.403839 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.403773    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"]
Jul 15 12:16:58.404334 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.404308    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" containerName="manager"
Jul 15 12:16:58.404334 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.404336    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" containerName="manager"
Jul 15 12:16:58.404482 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.404438    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" containerName="manager"
Jul 15 12:16:58.407820 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.407793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"
Jul 15 12:16:58.414633 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:58.414592    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:58.416361 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:58.416322    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:58.429525 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.429484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"]
Jul 15 12:16:58.445560 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:58.445510    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:58.505869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.505832    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac010e6-2051-4102-b3d2-408b37e0304e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89rvw\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.506057 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.505934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lsrx\" (UniqueName: \"kubernetes.io/projected/77742c0e-8679-4eaa-83ae-1c0e887b79d9-kube-api-access-6lsrx\") pod \"limitador-operator-controller-manager-859d7b78bd-rllfv\" (UID: \"77742c0e-8679-4eaa-83ae-1c0e887b79d9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"
Jul 15 12:16:58.506057 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.506021    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcpkk\" (UniqueName: \"kubernetes.io/projected/0ac010e6-2051-4102-b3d2-408b37e0304e-kube-api-access-tcpkk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89rvw\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.607172 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.607063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lsrx\" (UniqueName: \"kubernetes.io/projected/77742c0e-8679-4eaa-83ae-1c0e887b79d9-kube-api-access-6lsrx\") pod \"limitador-operator-controller-manager-859d7b78bd-rllfv\" (UID: \"77742c0e-8679-4eaa-83ae-1c0e887b79d9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"
Jul 15 12:16:58.607172 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.607149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpkk\" (UniqueName: \"kubernetes.io/projected/0ac010e6-2051-4102-b3d2-408b37e0304e-kube-api-access-tcpkk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89rvw\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.607408 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.607201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac010e6-2051-4102-b3d2-408b37e0304e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89rvw\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.607604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.607581    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac010e6-2051-4102-b3d2-408b37e0304e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89rvw\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.617824 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.617784    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcpkk\" (UniqueName: \"kubernetes.io/projected/0ac010e6-2051-4102-b3d2-408b37e0304e-kube-api-access-tcpkk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89rvw\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.617999 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.617841    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lsrx\" (UniqueName: \"kubernetes.io/projected/77742c0e-8679-4eaa-83ae-1c0e887b79d9-kube-api-access-6lsrx\") pod \"limitador-operator-controller-manager-859d7b78bd-rllfv\" (UID: \"77742c0e-8679-4eaa-83ae-1c0e887b79d9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"
Jul 15 12:16:58.763689 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.763644    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:16:58.768889 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:58.768696    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"
Jul 15 12:16:59.240723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.240687    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"]
Jul 15 12:16:59.251692 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.251660    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:16:59.254119 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:59.254066    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:59.260502 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.260429    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"]
Jul 15 12:16:59.279964 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:59.279915    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:59.312626 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.312576    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47003c97-c574-418d-9cd3-c892956c8a3d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5k6vn\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:16:59.312788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.312636    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mrv9\" (UniqueName: \"kubernetes.io/projected/47003c97-c574-418d-9cd3-c892956c8a3d-kube-api-access-2mrv9\") pod \"kuadrant-operator-controller-manager-86878c769f-5k6vn\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:16:59.327680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.327645    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rdr4n"
Jul 15 12:16:59.357739 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:59.357693    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:16:59.359342 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:16:59.359307    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:16:59.413486 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.413453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mrv9\" (UniqueName: \"kubernetes.io/projected/47003c97-c574-418d-9cd3-c892956c8a3d-kube-api-access-2mrv9\") pod \"kuadrant-operator-controller-manager-86878c769f-5k6vn\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:16:59.413665 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.413624    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47003c97-c574-418d-9cd3-c892956c8a3d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5k6vn\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:16:59.414024 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.414004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47003c97-c574-418d-9cd3-c892956c8a3d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5k6vn\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:16:59.428162 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.428132    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mrv9\" (UniqueName: \"kubernetes.io/projected/47003c97-c574-418d-9cd3-c892956c8a3d-kube-api-access-2mrv9\") pod \"kuadrant-operator-controller-manager-86878c769f-5k6vn\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:16:59.566981 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:16:59.566877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:17:00.062063 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:00.062011    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:00.068552 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:00.068069    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:17:10.970379 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:10.970352    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:10.978299 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:10.978231    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:17:10.998700 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:10.998450    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:11.003911 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.003865    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:11.007443 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.007403    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:17:11.030419 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.030396    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"]
Jul 15 12:17:11.032307 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:17:11.032276    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77742c0e_8679_4eaa_83ae_1c0e887b79d9.slice/crio-f8d05ea3e5a3158159f3c98ed0c4301f5cebbabaea945b9666e1cfffc2839662 WatchSource:0}: Error finding container f8d05ea3e5a3158159f3c98ed0c4301f5cebbabaea945b9666e1cfffc2839662: Status 404 returned error can't find the container with id f8d05ea3e5a3158159f3c98ed0c4301f5cebbabaea945b9666e1cfffc2839662
Jul 15 12:17:11.044904 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:17:11.044869    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47003c97_c574_418d_9cd3_c892956c8a3d.slice/crio-a82e938c86b5f1f934faefe0de9341f09118fb0a21e2db64003838a8ba45df8f WatchSource:0}: Error finding container a82e938c86b5f1f934faefe0de9341f09118fb0a21e2db64003838a8ba45df8f: Status 404 returned error can't find the container with id a82e938c86b5f1f934faefe0de9341f09118fb0a21e2db64003838a8ba45df8f
Jul 15 12:17:11.048924 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.048896    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"]
Jul 15 12:17:11.062024 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.062002    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"]
Jul 15 12:17:11.065816 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:17:11.065768    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac010e6_2051_4102_b3d2_408b37e0304e.slice/crio-d1fa7eefb7bf83be9c37f02afd2f11943efb2e8188403030ec629278a7307fbc WatchSource:0}: Error finding container d1fa7eefb7bf83be9c37f02afd2f11943efb2e8188403030ec629278a7307fbc: Status 404 returned error can't find the container with id d1fa7eefb7bf83be9c37f02afd2f11943efb2e8188403030ec629278a7307fbc
Jul 15 12:17:11.125648 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.125626    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ztqp\" (UniqueName: \"kubernetes.io/projected/d9d69db3-aab7-43c1-b5b1-675b353ec965-kube-api-access-6ztqp\") pod \"d9d69db3-aab7-43c1-b5b1-675b353ec965\" (UID: \"d9d69db3-aab7-43c1-b5b1-675b353ec965\") "
Jul 15 12:17:11.125778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.125685    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77pm\" (UniqueName: \"kubernetes.io/projected/b8d29787-cce7-447b-9f21-8abd816ed7da-kube-api-access-s77pm\") pod \"b8d29787-cce7-447b-9f21-8abd816ed7da\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") "
Jul 15 12:17:11.125778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.125720    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8d29787-cce7-447b-9f21-8abd816ed7da-extensions-socket-volume\") pod \"b8d29787-cce7-447b-9f21-8abd816ed7da\" (UID: \"b8d29787-cce7-447b-9f21-8abd816ed7da\") "
Jul 15 12:17:11.126197 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.126168    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8d29787-cce7-447b-9f21-8abd816ed7da-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b8d29787-cce7-447b-9f21-8abd816ed7da" (UID: "b8d29787-cce7-447b-9f21-8abd816ed7da"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:17:11.127567 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.127543    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d69db3-aab7-43c1-b5b1-675b353ec965-kube-api-access-6ztqp" (OuterVolumeSpecName: "kube-api-access-6ztqp") pod "d9d69db3-aab7-43c1-b5b1-675b353ec965" (UID: "d9d69db3-aab7-43c1-b5b1-675b353ec965"). InnerVolumeSpecName "kube-api-access-6ztqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:17:11.127729 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.127714    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d29787-cce7-447b-9f21-8abd816ed7da-kube-api-access-s77pm" (OuterVolumeSpecName: "kube-api-access-s77pm") pod "b8d29787-cce7-447b-9f21-8abd816ed7da" (UID: "b8d29787-cce7-447b-9f21-8abd816ed7da"). InnerVolumeSpecName "kube-api-access-s77pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:17:11.226540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.226504    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77pm\" (UniqueName: \"kubernetes.io/projected/b8d29787-cce7-447b-9f21-8abd816ed7da-kube-api-access-s77pm\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:11.226540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.226533    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8d29787-cce7-447b-9f21-8abd816ed7da-extensions-socket-volume\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:11.226540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.226544    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ztqp\" (UniqueName: \"kubernetes.io/projected/d9d69db3-aab7-43c1-b5b1-675b353ec965-kube-api-access-6ztqp\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:11.469671 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.469635    2582 generic.go:356] "Generic (PLEG): container finished" podID="d9d69db3-aab7-43c1-b5b1-675b353ec965" containerID="2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86" exitCode=0
Jul 15 12:17:11.469869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.469682    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:11.469869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.469734    2582 scope.go:117] "RemoveContainer" containerID="2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86"
Jul 15 12:17:11.471467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.471444    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw" event={"ID":"0ac010e6-2051-4102-b3d2-408b37e0304e","Type":"ContainerStarted","Data":"a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28"}
Jul 15 12:17:11.471582 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.471495    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw" event={"ID":"0ac010e6-2051-4102-b3d2-408b37e0304e","Type":"ContainerStarted","Data":"d1fa7eefb7bf83be9c37f02afd2f11943efb2e8188403030ec629278a7307fbc"}
Jul 15 12:17:11.471582 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.471518    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:17:11.472273 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.472238    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:17:11.473191 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.473166    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87" event={"ID":"598cede8-3607-497d-b472-43800fa6010e","Type":"ContainerStarted","Data":"09bcff1d53f6f305bf2348c9754dce5e252b0152ef71e26a9bfd83da93fe987a"}
Jul 15 12:17:11.474397 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.474315    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:11.474953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.474925    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv" event={"ID":"77742c0e-8679-4eaa-83ae-1c0e887b79d9","Type":"ContainerStarted","Data":"935939a0460cc6b7ed2feacac1717dee1d33d5b00efa9001508d4033ed2471ba"}
Jul 15 12:17:11.475041 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.474978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv" event={"ID":"77742c0e-8679-4eaa-83ae-1c0e887b79d9","Type":"ContainerStarted","Data":"f8d05ea3e5a3158159f3c98ed0c4301f5cebbabaea945b9666e1cfffc2839662"}
Jul 15 12:17:11.475041 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.475006    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"
Jul 15 12:17:11.476545 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.476516    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:17:11.476631 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.476541    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn" event={"ID":"47003c97-c574-418d-9cd3-c892956c8a3d","Type":"ContainerStarted","Data":"4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9"}
Jul 15 12:17:11.476631 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.476581    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn" event={"ID":"47003c97-c574-418d-9cd3-c892956c8a3d","Type":"ContainerStarted","Data":"a82e938c86b5f1f934faefe0de9341f09118fb0a21e2db64003838a8ba45df8f"}
Jul 15 12:17:11.476735 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.476654    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:17:11.477907 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.477886    2582 generic.go:356] "Generic (PLEG): container finished" podID="b8d29787-cce7-447b-9f21-8abd816ed7da" containerID="ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75" exitCode=0
Jul 15 12:17:11.477991 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.477923    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:17:11.480161 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.480140    2582 scope.go:117] "RemoveContainer" containerID="2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86"
Jul 15 12:17:11.480403 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.480378    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86\": container with ID starting with 2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86 not found: ID does not exist" containerID="2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86"
Jul 15 12:17:11.480464 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.480414    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86"} err="failed to get container status \"2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86\": rpc error: code = NotFound desc = could not find container \"2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86\": container with ID starting with 2e9ba236ff58e8f02530cf7ad70ffc4ae966dd57447aa64c4dfc0de7a1033f86 not found: ID does not exist"
Jul 15 12:17:11.480464 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.480435    2582 scope.go:117] "RemoveContainer" containerID="ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75"
Jul 15 12:17:11.489724 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.489703    2582 scope.go:117] "RemoveContainer" containerID="ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75"
Jul 15 12:17:11.490004 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.489985    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75\": container with ID starting with ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75 not found: ID does not exist" containerID="ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75"
Jul 15 12:17:11.490044 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.490013    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75"} err="failed to get container status \"ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75\": rpc error: code = NotFound desc = could not find container \"ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75\": container with ID starting with ce41c79b674ddcb6eccbdbe7634be6216e6208eb0e838cbfefe2456982a15f75 not found: ID does not exist"
Jul 15 12:17:11.497473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.497429    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s2n87" podStartSLOduration=1.507867498 podStartE2EDuration="24.497415001s" totalImagesPullingTime="22.989547503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:17:11.495429852 +0000 UTC m=+562.595379191" watchObservedRunningTime="2026-07-15 12:17:11.497415001 +0000 UTC m=+562.597364356"
Jul 15 12:17:11.498165 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.498141    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:11.529239 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.529192    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zmrwl\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zmrwl"
Jul 15 12:17:11.529239 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.529211    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw" podStartSLOduration=13.52919753 podStartE2EDuration="13.52919753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:17:11.526276301 +0000 UTC m=+562.626225642" watchObservedRunningTime="2026-07-15 12:17:11.52919753 +0000 UTC m=+562.629146869"
Jul 15 12:17:11.568554 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.568506    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv" podStartSLOduration=13.568490409 podStartE2EDuration="13.568490409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:17:11.566971586 +0000 UTC m=+562.666920925" watchObservedRunningTime="2026-07-15 12:17:11.568490409 +0000 UTC m=+562.668439747"
Jul 15 12:17:11.573182 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:11.573151    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-29f8x\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-29f8x"
Jul 15 12:17:11.609011 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:11.608958    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn" podStartSLOduration=12.608940817 podStartE2EDuration="12.608940817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:16:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:17:11.608039445 +0000 UTC m=+562.707988787" watchObservedRunningTime="2026-07-15 12:17:11.608940817 +0000 UTC m=+562.708890156"
Jul 15 12:17:12.062346 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:12.062270    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d29787-cce7-447b-9f21-8abd816ed7da" path="/var/lib/kubelet/pods/b8d29787-cce7-447b-9f21-8abd816ed7da/volumes"
Jul 15 12:17:12.062698 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:12.062585    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9d69db3-aab7-43c1-b5b1-675b353ec965" path="/var/lib/kubelet/pods/d9d69db3-aab7-43c1-b5b1-675b353ec965/volumes"
Jul 15 12:17:22.487153 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.487119    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:17:22.487604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.487204    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:17:22.487604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.487244    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rllfv"
Jul 15 12:17:22.567361 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.567330    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"]
Jul 15 12:17:22.567545 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.567521    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw" podUID="0ac010e6-2051-4102-b3d2-408b37e0304e" containerName="manager" containerID="cri-o://a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28" gracePeriod=10
Jul 15 12:17:22.821935 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.821901    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:17:22.830304 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.830277    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcpkk\" (UniqueName: \"kubernetes.io/projected/0ac010e6-2051-4102-b3d2-408b37e0304e-kube-api-access-tcpkk\") pod \"0ac010e6-2051-4102-b3d2-408b37e0304e\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") "
Jul 15 12:17:22.830448 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.830338    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac010e6-2051-4102-b3d2-408b37e0304e-extensions-socket-volume\") pod \"0ac010e6-2051-4102-b3d2-408b37e0304e\" (UID: \"0ac010e6-2051-4102-b3d2-408b37e0304e\") "
Jul 15 12:17:22.830739 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.830718    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac010e6-2051-4102-b3d2-408b37e0304e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ac010e6-2051-4102-b3d2-408b37e0304e" (UID: "0ac010e6-2051-4102-b3d2-408b37e0304e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:17:22.832476 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.832452    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac010e6-2051-4102-b3d2-408b37e0304e-kube-api-access-tcpkk" (OuterVolumeSpecName: "kube-api-access-tcpkk") pod "0ac010e6-2051-4102-b3d2-408b37e0304e" (UID: "0ac010e6-2051-4102-b3d2-408b37e0304e"). InnerVolumeSpecName "kube-api-access-tcpkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:17:22.931192 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.931150    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcpkk\" (UniqueName: \"kubernetes.io/projected/0ac010e6-2051-4102-b3d2-408b37e0304e-kube-api-access-tcpkk\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:22.931192 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:22.931183    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac010e6-2051-4102-b3d2-408b37e0304e-extensions-socket-volume\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:23.543329 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.543289    2582 generic.go:356] "Generic (PLEG): container finished" podID="0ac010e6-2051-4102-b3d2-408b37e0304e" containerID="a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28" exitCode=0
Jul 15 12:17:23.543800 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.543346    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"
Jul 15 12:17:23.543800 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.543355    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw" event={"ID":"0ac010e6-2051-4102-b3d2-408b37e0304e","Type":"ContainerDied","Data":"a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28"}
Jul 15 12:17:23.543800 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.543447    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw" event={"ID":"0ac010e6-2051-4102-b3d2-408b37e0304e","Type":"ContainerDied","Data":"d1fa7eefb7bf83be9c37f02afd2f11943efb2e8188403030ec629278a7307fbc"}
Jul 15 12:17:23.543800 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.543482    2582 scope.go:117] "RemoveContainer" containerID="a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28"
Jul 15 12:17:23.553021 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.553002    2582 scope.go:117] "RemoveContainer" containerID="a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28"
Jul 15 12:17:23.553332 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:23.553314    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28\": container with ID starting with a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28 not found: ID does not exist" containerID="a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28"
Jul 15 12:17:23.553414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.553339    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28"} err="failed to get container status \"a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28\": rpc error: code = NotFound desc = could not find container \"a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28\": container with ID starting with a7527f884e57e7d7873933a57c0deb19c34ac57b55dcc63dcbb5c8ca6d7fce28 not found: ID does not exist"
Jul 15 12:17:23.569892 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.569848    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"]
Jul 15 12:17:23.581709 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:23.581666    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89rvw"]
Jul 15 12:17:24.061965 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:24.061934    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac010e6-2051-4102-b3d2-408b37e0304e" path="/var/lib/kubelet/pods/0ac010e6-2051-4102-b3d2-408b37e0304e/volumes"
Jul 15 12:17:41.652047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.652008    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k84hm"]
Jul 15 12:17:41.652630 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.652535    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac010e6-2051-4102-b3d2-408b37e0304e" containerName="manager"
Jul 15 12:17:41.652630 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.652555    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac010e6-2051-4102-b3d2-408b37e0304e" containerName="manager"
Jul 15 12:17:41.652745 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.652666    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac010e6-2051-4102-b3d2-408b37e0304e" containerName="manager"
Jul 15 12:17:41.711227 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.711187    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k84hm"]
Jul 15 12:17:41.711409 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.711310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:41.714407 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.714381    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 12:17:41.760636 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.760604    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k84hm"]
Jul 15 12:17:41.774414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.774387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2vpx\" (UniqueName: \"kubernetes.io/projected/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-kube-api-access-m2vpx\") pod \"limitador-limitador-64c94fd64f-k84hm\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:41.774585 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.774461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-config-file\") pod \"limitador-limitador-64c94fd64f-k84hm\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:41.875320 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.875279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-config-file\") pod \"limitador-limitador-64c94fd64f-k84hm\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:41.875513 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.875341    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2vpx\" (UniqueName: \"kubernetes.io/projected/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-kube-api-access-m2vpx\") pod \"limitador-limitador-64c94fd64f-k84hm\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:41.875921 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.875898    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-config-file\") pod \"limitador-limitador-64c94fd64f-k84hm\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:41.886972 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:41.886945    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2vpx\" (UniqueName: \"kubernetes.io/projected/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-kube-api-access-m2vpx\") pod \"limitador-limitador-64c94fd64f-k84hm\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:42.024490 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.024456    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:42.166884 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.166861    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k84hm"]
Jul 15 12:17:42.172142 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:17:42.172056    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a9a2e1_e695_4373_9fbf_2df9b71f0247.slice/crio-288d6011532228a202d768fc082c29b3de2f1fb8ef6f75041b56cb069f0060aa WatchSource:0}: Error finding container 288d6011532228a202d768fc082c29b3de2f1fb8ef6f75041b56cb069f0060aa: Status 404 returned error can't find the container with id 288d6011532228a202d768fc082c29b3de2f1fb8ef6f75041b56cb069f0060aa
Jul 15 12:17:42.612341 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.612301    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-9wqq9"]
Jul 15 12:17:42.617273 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.617240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9wqq9"
Jul 15 12:17:42.620771 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.620747    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fhqfb\""
Jul 15 12:17:42.629484 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.629456    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9wqq9"]
Jul 15 12:17:42.646219 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.646182    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm" event={"ID":"c2a9a2e1-e695-4373-9fbf-2df9b71f0247","Type":"ContainerStarted","Data":"288d6011532228a202d768fc082c29b3de2f1fb8ef6f75041b56cb069f0060aa"}
Jul 15 12:17:42.680764 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.680728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmp5q\" (UniqueName: \"kubernetes.io/projected/8528fb6d-9ce1-4e66-88b9-3e331bab31f2-kube-api-access-tmp5q\") pod \"authorino-58cd955675-9wqq9\" (UID: \"8528fb6d-9ce1-4e66-88b9-3e331bab31f2\") " pod="kuadrant-system/authorino-58cd955675-9wqq9"
Jul 15 12:17:42.781628 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.781584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmp5q\" (UniqueName: \"kubernetes.io/projected/8528fb6d-9ce1-4e66-88b9-3e331bab31f2-kube-api-access-tmp5q\") pod \"authorino-58cd955675-9wqq9\" (UID: \"8528fb6d-9ce1-4e66-88b9-3e331bab31f2\") " pod="kuadrant-system/authorino-58cd955675-9wqq9"
Jul 15 12:17:42.791325 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.791287    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmp5q\" (UniqueName: \"kubernetes.io/projected/8528fb6d-9ce1-4e66-88b9-3e331bab31f2-kube-api-access-tmp5q\") pod \"authorino-58cd955675-9wqq9\" (UID: \"8528fb6d-9ce1-4e66-88b9-3e331bab31f2\") " pod="kuadrant-system/authorino-58cd955675-9wqq9"
Jul 15 12:17:42.917711 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.915767    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hjp4"]
Jul 15 12:17:42.920967 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.920941    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hjp4"
Jul 15 12:17:42.928193 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.927851    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9wqq9"
Jul 15 12:17:42.930218 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.929557    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hjp4"]
Jul 15 12:17:42.983797 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:42.983693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kvqq\" (UniqueName: \"kubernetes.io/projected/671255a1-8242-493d-8eb3-760b978433f1-kube-api-access-2kvqq\") pod \"authorino-75d556bd77-7hjp4\" (UID: \"671255a1-8242-493d-8eb3-760b978433f1\") " pod="kuadrant-system/authorino-75d556bd77-7hjp4"
Jul 15 12:17:43.084792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:43.084752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kvqq\" (UniqueName: \"kubernetes.io/projected/671255a1-8242-493d-8eb3-760b978433f1-kube-api-access-2kvqq\") pod \"authorino-75d556bd77-7hjp4\" (UID: \"671255a1-8242-493d-8eb3-760b978433f1\") " pod="kuadrant-system/authorino-75d556bd77-7hjp4"
Jul 15 12:17:43.098086 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:43.098051    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kvqq\" (UniqueName: \"kubernetes.io/projected/671255a1-8242-493d-8eb3-760b978433f1-kube-api-access-2kvqq\") pod \"authorino-75d556bd77-7hjp4\" (UID: \"671255a1-8242-493d-8eb3-760b978433f1\") " pod="kuadrant-system/authorino-75d556bd77-7hjp4"
Jul 15 12:17:43.134087 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:43.134047    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9wqq9"]
Jul 15 12:17:43.136863 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:17:43.136833    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8528fb6d_9ce1_4e66_88b9_3e331bab31f2.slice/crio-d03f691364e66450aa9a458f47127945c3659a326607d8c30b70cc86403fef6a WatchSource:0}: Error finding container d03f691364e66450aa9a458f47127945c3659a326607d8c30b70cc86403fef6a: Status 404 returned error can't find the container with id d03f691364e66450aa9a458f47127945c3659a326607d8c30b70cc86403fef6a
Jul 15 12:17:43.238838 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:43.238802    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hjp4"
Jul 15 12:17:43.410465 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:43.410436    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hjp4"]
Jul 15 12:17:43.412889 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:17:43.412857    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod671255a1_8242_493d_8eb3_760b978433f1.slice/crio-2d6feaca0aafdf52a20f24e616291d0c34ebfb8a561a300a4816e415d3e4644c WatchSource:0}: Error finding container 2d6feaca0aafdf52a20f24e616291d0c34ebfb8a561a300a4816e415d3e4644c: Status 404 returned error can't find the container with id 2d6feaca0aafdf52a20f24e616291d0c34ebfb8a561a300a4816e415d3e4644c
Jul 15 12:17:43.658206 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:43.658118    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hjp4" event={"ID":"671255a1-8242-493d-8eb3-760b978433f1","Type":"ContainerStarted","Data":"2d6feaca0aafdf52a20f24e616291d0c34ebfb8a561a300a4816e415d3e4644c"}
Jul 15 12:17:43.660142 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:43.660085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9wqq9" event={"ID":"8528fb6d-9ce1-4e66-88b9-3e331bab31f2","Type":"ContainerStarted","Data":"d03f691364e66450aa9a458f47127945c3659a326607d8c30b70cc86403fef6a"}
Jul 15 12:17:45.683896 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:45.683853    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm" event={"ID":"c2a9a2e1-e695-4373-9fbf-2df9b71f0247","Type":"ContainerStarted","Data":"50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1"}
Jul 15 12:17:45.684404 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:45.683993    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:45.704151 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:45.703939    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm" podStartSLOduration=1.780531415 podStartE2EDuration="4.703916167s" totalImagesPullingTime="2.923384752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:17:45.70211471 +0000 UTC m=+596.802064046" watchObservedRunningTime="2026-07-15 12:17:45.703916167 +0000 UTC m=+596.803865505"
Jul 15 12:17:47.696566 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:47.696511    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hjp4" event={"ID":"671255a1-8242-493d-8eb3-760b978433f1","Type":"ContainerStarted","Data":"5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc"}
Jul 15 12:17:47.698306 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:47.698265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9wqq9" event={"ID":"8528fb6d-9ce1-4e66-88b9-3e331bab31f2","Type":"ContainerStarted","Data":"0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff"}
Jul 15 12:17:47.714862 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:47.714793    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7hjp4" podStartSLOduration=1.5584170149999999 podStartE2EDuration="5.714771984s" totalImagesPullingTime="4.156354969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:17:47.713959536 +0000 UTC m=+598.813908887" watchObservedRunningTime="2026-07-15 12:17:47.714771984 +0000 UTC m=+598.814721317"
Jul 15 12:17:47.731857 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:47.731567    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-9wqq9" podStartSLOduration=1.294589614 podStartE2EDuration="5.73154439s" totalImagesPullingTime="4.436954776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:17:47.730029178 +0000 UTC m=+598.829978517" watchObservedRunningTime="2026-07-15 12:17:47.73154439 +0000 UTC m=+598.831493730"
Jul 15 12:17:47.767810 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:47.767769    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9wqq9"]
Jul 15 12:17:49.375860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:49.375830    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:17:49.376315 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:49.375977    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:17:49.708798 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:49.708734    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-9wqq9" podUID="8528fb6d-9ce1-4e66-88b9-3e331bab31f2" containerName="authorino" containerID="cri-o://0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff" gracePeriod=30
Jul 15 12:17:49.880737 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:49.880683    2582 info.go:109] Failed to get network devices: open /sys/class/net/d03f691364e6645/address: no such file or directory
Jul 15 12:17:49.954732 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:49.954706    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9wqq9"
Jul 15 12:17:50.047062 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.046963    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmp5q\" (UniqueName: \"kubernetes.io/projected/8528fb6d-9ce1-4e66-88b9-3e331bab31f2-kube-api-access-tmp5q\") pod \"8528fb6d-9ce1-4e66-88b9-3e331bab31f2\" (UID: \"8528fb6d-9ce1-4e66-88b9-3e331bab31f2\") "
Jul 15 12:17:50.049170 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.049139    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8528fb6d-9ce1-4e66-88b9-3e331bab31f2-kube-api-access-tmp5q" (OuterVolumeSpecName: "kube-api-access-tmp5q") pod "8528fb6d-9ce1-4e66-88b9-3e331bab31f2" (UID: "8528fb6d-9ce1-4e66-88b9-3e331bab31f2"). InnerVolumeSpecName "kube-api-access-tmp5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:17:50.147983 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.147944    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmp5q\" (UniqueName: \"kubernetes.io/projected/8528fb6d-9ce1-4e66-88b9-3e331bab31f2-kube-api-access-tmp5q\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:50.715952 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.715914    2582 generic.go:356] "Generic (PLEG): container finished" podID="8528fb6d-9ce1-4e66-88b9-3e331bab31f2" containerID="0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff" exitCode=0
Jul 15 12:17:50.715952 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.715957    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9wqq9"
Jul 15 12:17:50.716507 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.715980    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9wqq9" event={"ID":"8528fb6d-9ce1-4e66-88b9-3e331bab31f2","Type":"ContainerDied","Data":"0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff"}
Jul 15 12:17:50.716507 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.716018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9wqq9" event={"ID":"8528fb6d-9ce1-4e66-88b9-3e331bab31f2","Type":"ContainerDied","Data":"d03f691364e66450aa9a458f47127945c3659a326607d8c30b70cc86403fef6a"}
Jul 15 12:17:50.716507 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.716042    2582 scope.go:117] "RemoveContainer" containerID="0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff"
Jul 15 12:17:50.725122 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.725085    2582 scope.go:117] "RemoveContainer" containerID="0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff"
Jul 15 12:17:50.725420 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:50.725399    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff\": container with ID starting with 0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff not found: ID does not exist" containerID="0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff"
Jul 15 12:17:50.725470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.725430    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff"} err="failed to get container status \"0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff\": rpc error: code = NotFound desc = could not find container \"0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff\": container with ID starting with 0e01bcbb2c91df93405256aad30a430b29d34c3321809c8f7c1344e8785f84ff not found: ID does not exist"
Jul 15 12:17:50.736974 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.736943    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9wqq9"]
Jul 15 12:17:50.739568 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:50.739544    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-9wqq9"]
Jul 15 12:17:52.062870 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:52.062829    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8528fb6d-9ce1-4e66-88b9-3e331bab31f2" path="/var/lib/kubelet/pods/8528fb6d-9ce1-4e66-88b9-3e331bab31f2/volumes"
Jul 15 12:17:56.691585 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:56.691554    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:57.521723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:57.521680    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k84hm"]
Jul 15 12:17:57.521946 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:57.521920    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm" podUID="c2a9a2e1-e695-4373-9fbf-2df9b71f0247" containerName="limitador" containerID="cri-o://50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1" gracePeriod=30
Jul 15 12:17:58.075264 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.075238    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:58.111342 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.111259    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2vpx\" (UniqueName: \"kubernetes.io/projected/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-kube-api-access-m2vpx\") pod \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") "
Jul 15 12:17:58.111342 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.111305    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-config-file\") pod \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\" (UID: \"c2a9a2e1-e695-4373-9fbf-2df9b71f0247\") "
Jul 15 12:17:58.111696 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.111669    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-config-file" (OuterVolumeSpecName: "config-file") pod "c2a9a2e1-e695-4373-9fbf-2df9b71f0247" (UID: "c2a9a2e1-e695-4373-9fbf-2df9b71f0247"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:17:58.113497 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.113465    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-kube-api-access-m2vpx" (OuterVolumeSpecName: "kube-api-access-m2vpx") pod "c2a9a2e1-e695-4373-9fbf-2df9b71f0247" (UID: "c2a9a2e1-e695-4373-9fbf-2df9b71f0247"). InnerVolumeSpecName "kube-api-access-m2vpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:17:58.212058 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.212025    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2vpx\" (UniqueName: \"kubernetes.io/projected/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-kube-api-access-m2vpx\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:58.212058 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.212055    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a9a2e1-e695-4373-9fbf-2df9b71f0247-config-file\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:17:58.757546 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.757511    2582 generic.go:356] "Generic (PLEG): container finished" podID="c2a9a2e1-e695-4373-9fbf-2df9b71f0247" containerID="50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1" exitCode=0
Jul 15 12:17:58.757718 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.757552    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm"
Jul 15 12:17:58.757718 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.757556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm" event={"ID":"c2a9a2e1-e695-4373-9fbf-2df9b71f0247","Type":"ContainerDied","Data":"50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1"}
Jul 15 12:17:58.757718 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.757686    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k84hm" event={"ID":"c2a9a2e1-e695-4373-9fbf-2df9b71f0247","Type":"ContainerDied","Data":"288d6011532228a202d768fc082c29b3de2f1fb8ef6f75041b56cb069f0060aa"}
Jul 15 12:17:58.757718 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.757717    2582 scope.go:117] "RemoveContainer" containerID="50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1"
Jul 15 12:17:58.766668 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.766650    2582 scope.go:117] "RemoveContainer" containerID="50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1"
Jul 15 12:17:58.766929 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:17:58.766914    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1\": container with ID starting with 50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1 not found: ID does not exist" containerID="50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1"
Jul 15 12:17:58.767002 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.766935    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1"} err="failed to get container status \"50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1\": rpc error: code = NotFound desc = could not find container \"50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1\": container with ID starting with 50b47fa0cc0422f36f689df4f181423762deabbd6a36163fc0ad99a4f73bdaf1 not found: ID does not exist"
Jul 15 12:17:58.786490 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.786456    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k84hm"]
Jul 15 12:17:58.788439 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:17:58.788417    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k84hm"]
Jul 15 12:18:00.063456 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:00.063421    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2a9a2e1-e695-4373-9fbf-2df9b71f0247" path="/var/lib/kubelet/pods/c2a9a2e1-e695-4373-9fbf-2df9b71f0247/volumes"
Jul 15 12:18:03.399762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.399724    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"]
Jul 15 12:18:03.400286 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.400265    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a9a2e1-e695-4373-9fbf-2df9b71f0247" containerName="limitador"
Jul 15 12:18:03.400286 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.400287    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a9a2e1-e695-4373-9fbf-2df9b71f0247" containerName="limitador"
Jul 15 12:18:03.400383 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.400298    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8528fb6d-9ce1-4e66-88b9-3e331bab31f2" containerName="authorino"
Jul 15 12:18:03.400383 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.400305    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8528fb6d-9ce1-4e66-88b9-3e331bab31f2" containerName="authorino"
Jul 15 12:18:03.400383 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.400370    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8528fb6d-9ce1-4e66-88b9-3e331bab31f2" containerName="authorino"
Jul 15 12:18:03.400383 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.400381    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2a9a2e1-e695-4373-9fbf-2df9b71f0247" containerName="limitador"
Jul 15 12:18:03.403020 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.403003    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.405365 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.405342    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 12:18:03.405502 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.405342    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 12:18:03.405502 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.405448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ttvzp\""
Jul 15 12:18:03.405604 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.405551    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 12:18:03.412672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.412644    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"]
Jul 15 12:18:03.452944 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.452909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wdsr\" (UniqueName: \"kubernetes.io/projected/60c0567b-a26c-42ad-b284-088f95564c76-kube-api-access-4wdsr\") pod \"postgres-69fb4594cf-l9kbp\" (UID: \"60c0567b-a26c-42ad-b284-088f95564c76\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.453156 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.452959    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/60c0567b-a26c-42ad-b284-088f95564c76-data\") pod \"postgres-69fb4594cf-l9kbp\" (UID: \"60c0567b-a26c-42ad-b284-088f95564c76\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.554414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.554379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/60c0567b-a26c-42ad-b284-088f95564c76-data\") pod \"postgres-69fb4594cf-l9kbp\" (UID: \"60c0567b-a26c-42ad-b284-088f95564c76\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.554556 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.554462    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wdsr\" (UniqueName: \"kubernetes.io/projected/60c0567b-a26c-42ad-b284-088f95564c76-kube-api-access-4wdsr\") pod \"postgres-69fb4594cf-l9kbp\" (UID: \"60c0567b-a26c-42ad-b284-088f95564c76\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.554769 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.554748    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/60c0567b-a26c-42ad-b284-088f95564c76-data\") pod \"postgres-69fb4594cf-l9kbp\" (UID: \"60c0567b-a26c-42ad-b284-088f95564c76\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.565299 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.565274    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wdsr\" (UniqueName: \"kubernetes.io/projected/60c0567b-a26c-42ad-b284-088f95564c76-kube-api-access-4wdsr\") pod \"postgres-69fb4594cf-l9kbp\" (UID: \"60c0567b-a26c-42ad-b284-088f95564c76\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.713788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.713751    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:03.859821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:03.859724    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"]
Jul 15 12:18:03.862708 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:03.862683    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c0567b_a26c_42ad_b284_088f95564c76.slice/crio-130a73cf6450ad0cb77ec7aa37e8b1d950253d6583cbd88ccbfc8123c7b62f54 WatchSource:0}: Error finding container 130a73cf6450ad0cb77ec7aa37e8b1d950253d6583cbd88ccbfc8123c7b62f54: Status 404 returned error can't find the container with id 130a73cf6450ad0cb77ec7aa37e8b1d950253d6583cbd88ccbfc8123c7b62f54
Jul 15 12:18:04.797339 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:04.797297    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp" event={"ID":"60c0567b-a26c-42ad-b284-088f95564c76","Type":"ContainerStarted","Data":"130a73cf6450ad0cb77ec7aa37e8b1d950253d6583cbd88ccbfc8123c7b62f54"}
Jul 15 12:18:08.822016 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:08.821984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp" event={"ID":"60c0567b-a26c-42ad-b284-088f95564c76","Type":"ContainerStarted","Data":"cb68ca5a333ed6afa83af19717a9ae53a20d8c9caf8fe337246fc10d4d40aef6"}
Jul 15 12:18:08.822420 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:08.822058    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:08.848369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:08.848314    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp" podStartSLOduration=1.080715848 podStartE2EDuration="5.848298045s" totalImagesPullingTime="4.767582197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:18:08.846832282 +0000 UTC m=+619.946781635" watchObservedRunningTime="2026-07-15 12:18:08.848298045 +0000 UTC m=+619.948247384"
Jul 15 12:18:14.858331 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:14.858251    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9kbp"
Jul 15 12:18:15.524147 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.524115    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fwt9n"]
Jul 15 12:18:15.527631 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.527611    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fwt9n"
Jul 15 12:18:15.549388 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.549349    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fwt9n"]
Jul 15 12:18:15.662607 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.662572    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxhwd\" (UniqueName: \"kubernetes.io/projected/7ef9e20c-4e41-4722-9eed-f27b5b2af5ac-kube-api-access-cxhwd\") pod \"authorino-5d4b67c446-fwt9n\" (UID: \"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac\") " pod="kuadrant-system/authorino-5d4b67c446-fwt9n"
Jul 15 12:18:15.763462 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.763421    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxhwd\" (UniqueName: \"kubernetes.io/projected/7ef9e20c-4e41-4722-9eed-f27b5b2af5ac-kube-api-access-cxhwd\") pod \"authorino-5d4b67c446-fwt9n\" (UID: \"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac\") " pod="kuadrant-system/authorino-5d4b67c446-fwt9n"
Jul 15 12:18:15.776912 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.776831    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxhwd\" (UniqueName: \"kubernetes.io/projected/7ef9e20c-4e41-4722-9eed-f27b5b2af5ac-kube-api-access-cxhwd\") pod \"authorino-5d4b67c446-fwt9n\" (UID: \"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac\") " pod="kuadrant-system/authorino-5d4b67c446-fwt9n"
Jul 15 12:18:15.808733 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.808695    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fwt9n"]
Jul 15 12:18:15.808943 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.808930    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fwt9n"
Jul 15 12:18:15.847869 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.847840    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c7485bc96-5pbwg"]
Jul 15 12:18:15.852023 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.851992    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c7485bc96-5pbwg"
Jul 15 12:18:15.873655 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.873616    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c7485bc96-5pbwg"]
Jul 15 12:18:15.965290 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.965250    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvxf\" (UniqueName: \"kubernetes.io/projected/aec5dd5f-a1d0-4a90-a11a-575839a4d2c0-kube-api-access-whvxf\") pod \"authorino-6c7485bc96-5pbwg\" (UID: \"aec5dd5f-a1d0-4a90-a11a-575839a4d2c0\") " pod="kuadrant-system/authorino-6c7485bc96-5pbwg"
Jul 15 12:18:15.974807 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:15.974767    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fwt9n"]
Jul 15 12:18:15.975865 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:15.975837    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ef9e20c_4e41_4722_9eed_f27b5b2af5ac.slice/crio-a200c026c46bc376b3fb932de15dfd9349e15e7bde26f4d4d452f4d59dee23e2 WatchSource:0}: Error finding container a200c026c46bc376b3fb932de15dfd9349e15e7bde26f4d4d452f4d59dee23e2: Status 404 returned error can't find the container with id a200c026c46bc376b3fb932de15dfd9349e15e7bde26f4d4d452f4d59dee23e2
Jul 15 12:18:16.012672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.012628    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c7485bc96-5pbwg"]
Jul 15 12:18:16.012940 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:16.012913    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-whvxf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6c7485bc96-5pbwg" podUID="aec5dd5f-a1d0-4a90-a11a-575839a4d2c0"
Jul 15 12:18:16.063715 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.063628    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cc48ff444-pxspr"]
Jul 15 12:18:16.066150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.066122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvxf\" (UniqueName: \"kubernetes.io/projected/aec5dd5f-a1d0-4a90-a11a-575839a4d2c0-kube-api-access-whvxf\") pod \"authorino-6c7485bc96-5pbwg\" (UID: \"aec5dd5f-a1d0-4a90-a11a-575839a4d2c0\") " pod="kuadrant-system/authorino-6c7485bc96-5pbwg"
Jul 15 12:18:16.067016 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.067002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.069868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.069852    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 15 12:18:16.079618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.079584    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cc48ff444-pxspr"]
Jul 15 12:18:16.081688 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.081662    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvxf\" (UniqueName: \"kubernetes.io/projected/aec5dd5f-a1d0-4a90-a11a-575839a4d2c0-kube-api-access-whvxf\") pod \"authorino-6c7485bc96-5pbwg\" (UID: \"aec5dd5f-a1d0-4a90-a11a-575839a4d2c0\") " pod="kuadrant-system/authorino-6c7485bc96-5pbwg"
Jul 15 12:18:16.167445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.167399    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s4zv\" (UniqueName: \"kubernetes.io/projected/0d4ac582-d212-4342-bebc-b3afc5f20bd7-kube-api-access-4s4zv\") pod \"authorino-cc48ff444-pxspr\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") " pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.167624 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.167486    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d4ac582-d212-4342-bebc-b3afc5f20bd7-tls-cert\") pod \"authorino-cc48ff444-pxspr\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") " pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.268392 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.268359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s4zv\" (UniqueName: \"kubernetes.io/projected/0d4ac582-d212-4342-bebc-b3afc5f20bd7-kube-api-access-4s4zv\") pod \"authorino-cc48ff444-pxspr\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") " pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.268568 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.268429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d4ac582-d212-4342-bebc-b3afc5f20bd7-tls-cert\") pod \"authorino-cc48ff444-pxspr\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") " pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.270748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.270727    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d4ac582-d212-4342-bebc-b3afc5f20bd7-tls-cert\") pod \"authorino-cc48ff444-pxspr\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") " pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.279540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.279515    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s4zv\" (UniqueName: \"kubernetes.io/projected/0d4ac582-d212-4342-bebc-b3afc5f20bd7-kube-api-access-4s4zv\") pod \"authorino-cc48ff444-pxspr\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") " pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.376891 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.376858    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:18:16.525242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.525214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cc48ff444-pxspr"]
Jul 15 12:18:16.527495 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:16.527469    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d4ac582_d212_4342_bebc_b3afc5f20bd7.slice/crio-ee2975a8735ea5686828d4b0738b6aff295c3b88bc4d0dece696ef31ec28e6b0 WatchSource:0}: Error finding container ee2975a8735ea5686828d4b0738b6aff295c3b88bc4d0dece696ef31ec28e6b0: Status 404 returned error can't find the container with id ee2975a8735ea5686828d4b0738b6aff295c3b88bc4d0dece696ef31ec28e6b0
Jul 15 12:18:16.866038 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.866000    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc48ff444-pxspr" event={"ID":"0d4ac582-d212-4342-bebc-b3afc5f20bd7","Type":"ContainerStarted","Data":"ee2975a8735ea5686828d4b0738b6aff295c3b88bc4d0dece696ef31ec28e6b0"}
Jul 15 12:18:16.867441 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.867409    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fwt9n" event={"ID":"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac","Type":"ContainerStarted","Data":"a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c"}
Jul 15 12:18:16.867441 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.867435    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c7485bc96-5pbwg"
Jul 15 12:18:16.867645 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.867437    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fwt9n" podUID="7ef9e20c-4e41-4722-9eed-f27b5b2af5ac" containerName="authorino" containerID="cri-o://a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c" gracePeriod=30
Jul 15 12:18:16.867645 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.867449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fwt9n" event={"ID":"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac","Type":"ContainerStarted","Data":"a200c026c46bc376b3fb932de15dfd9349e15e7bde26f4d4d452f4d59dee23e2"}
Jul 15 12:18:16.876374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.876346    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c7485bc96-5pbwg"
Jul 15 12:18:16.890787 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.890737    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fwt9n" podStartSLOduration=1.551308804 podStartE2EDuration="1.890719027s" totalImagesPullingTime="339.410223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:18:16.888210449 +0000 UTC m=+627.988159799" watchObservedRunningTime="2026-07-15 12:18:16.890719027 +0000 UTC m=+627.990668366"
Jul 15 12:18:16.974757 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.974494    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whvxf\" (UniqueName: \"kubernetes.io/projected/aec5dd5f-a1d0-4a90-a11a-575839a4d2c0-kube-api-access-whvxf\") pod \"aec5dd5f-a1d0-4a90-a11a-575839a4d2c0\" (UID: \"aec5dd5f-a1d0-4a90-a11a-575839a4d2c0\") "
Jul 15 12:18:16.977031 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:16.977002    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec5dd5f-a1d0-4a90-a11a-575839a4d2c0-kube-api-access-whvxf" (OuterVolumeSpecName: "kube-api-access-whvxf") pod "aec5dd5f-a1d0-4a90-a11a-575839a4d2c0" (UID: "aec5dd5f-a1d0-4a90-a11a-575839a4d2c0"). InnerVolumeSpecName "kube-api-access-whvxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:18:17.075342 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.075306    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whvxf\" (UniqueName: \"kubernetes.io/projected/aec5dd5f-a1d0-4a90-a11a-575839a4d2c0-kube-api-access-whvxf\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:18:17.134152 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.134121    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fwt9n"
Jul 15 12:18:17.277312 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.277260    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxhwd\" (UniqueName: \"kubernetes.io/projected/7ef9e20c-4e41-4722-9eed-f27b5b2af5ac-kube-api-access-cxhwd\") pod \"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac\" (UID: \"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac\") "
Jul 15 12:18:17.279316 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.279292    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef9e20c-4e41-4722-9eed-f27b5b2af5ac-kube-api-access-cxhwd" (OuterVolumeSpecName: "kube-api-access-cxhwd") pod "7ef9e20c-4e41-4722-9eed-f27b5b2af5ac" (UID: "7ef9e20c-4e41-4722-9eed-f27b5b2af5ac"). InnerVolumeSpecName "kube-api-access-cxhwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:18:17.378747 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.378654    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxhwd\" (UniqueName: \"kubernetes.io/projected/7ef9e20c-4e41-4722-9eed-f27b5b2af5ac-kube-api-access-cxhwd\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:18:17.873583 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.873546    2582 generic.go:356] "Generic (PLEG): container finished" podID="7ef9e20c-4e41-4722-9eed-f27b5b2af5ac" containerID="a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c" exitCode=0
Jul 15 12:18:17.873583 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.873584    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fwt9n"
Jul 15 12:18:17.873823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.873628    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fwt9n" event={"ID":"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac","Type":"ContainerDied","Data":"a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c"}
Jul 15 12:18:17.873823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.873672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fwt9n" event={"ID":"7ef9e20c-4e41-4722-9eed-f27b5b2af5ac","Type":"ContainerDied","Data":"a200c026c46bc376b3fb932de15dfd9349e15e7bde26f4d4d452f4d59dee23e2"}
Jul 15 12:18:17.873823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.873691    2582 scope.go:117] "RemoveContainer" containerID="a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c"
Jul 15 12:18:17.876613 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.876587    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c7485bc96-5pbwg"
Jul 15 12:18:17.877458 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.877432    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc48ff444-pxspr" event={"ID":"0d4ac582-d212-4342-bebc-b3afc5f20bd7","Type":"ContainerStarted","Data":"4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e"}
Jul 15 12:18:17.888821 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.888802    2582 scope.go:117] "RemoveContainer" containerID="a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c"
Jul 15 12:18:17.889138 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:17.889117    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c\": container with ID starting with a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c not found: ID does not exist" containerID="a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c"
Jul 15 12:18:17.889213 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.889147    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c"} err="failed to get container status \"a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c\": rpc error: code = NotFound desc = could not find container \"a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c\": container with ID starting with a40aa65e7b9846efecce9cfa10c670f3ba53e9637ef2fc3faef7a374f2775a5c not found: ID does not exist"
Jul 15 12:18:17.907006 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.906946    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cc48ff444-pxspr" podStartSLOduration=1.533503533 podStartE2EDuration="1.906928118s" totalImagesPullingTime="373.424585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:18:17.905227186 +0000 UTC m=+629.005176526" watchObservedRunningTime="2026-07-15 12:18:17.906928118 +0000 UTC m=+629.006877457"
Jul 15 12:18:17.946758 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.946720    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hjp4"]
Jul 15 12:18:17.946986 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.946961    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7hjp4" podUID="671255a1-8242-493d-8eb3-760b978433f1" containerName="authorino" containerID="cri-o://5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc" gracePeriod=30
Jul 15 12:18:17.952555 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.952524    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c7485bc96-5pbwg"]
Jul 15 12:18:17.962015 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.961976    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c7485bc96-5pbwg"]
Jul 15 12:18:17.976662 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.976622    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fwt9n"]
Jul 15 12:18:17.986169 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:17.986131    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fwt9n"]
Jul 15 12:18:18.062771 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.062699    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ef9e20c-4e41-4722-9eed-f27b5b2af5ac" path="/var/lib/kubelet/pods/7ef9e20c-4e41-4722-9eed-f27b5b2af5ac/volumes"
Jul 15 12:18:18.063018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.063006    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aec5dd5f-a1d0-4a90-a11a-575839a4d2c0" path="/var/lib/kubelet/pods/aec5dd5f-a1d0-4a90-a11a-575839a4d2c0/volumes"
Jul 15 12:18:18.256942 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.256912    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hjp4"
Jul 15 12:18:18.387414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.387335    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kvqq\" (UniqueName: \"kubernetes.io/projected/671255a1-8242-493d-8eb3-760b978433f1-kube-api-access-2kvqq\") pod \"671255a1-8242-493d-8eb3-760b978433f1\" (UID: \"671255a1-8242-493d-8eb3-760b978433f1\") "
Jul 15 12:18:18.389569 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.389534    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/671255a1-8242-493d-8eb3-760b978433f1-kube-api-access-2kvqq" (OuterVolumeSpecName: "kube-api-access-2kvqq") pod "671255a1-8242-493d-8eb3-760b978433f1" (UID: "671255a1-8242-493d-8eb3-760b978433f1"). InnerVolumeSpecName "kube-api-access-2kvqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:18:18.488195 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.488154    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kvqq\" (UniqueName: \"kubernetes.io/projected/671255a1-8242-493d-8eb3-760b978433f1-kube-api-access-2kvqq\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:18:18.884353 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.884315    2582 generic.go:356] "Generic (PLEG): container finished" podID="671255a1-8242-493d-8eb3-760b978433f1" containerID="5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc" exitCode=0
Jul 15 12:18:18.884798 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.884360    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7hjp4"
Jul 15 12:18:18.884798 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.884401    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hjp4" event={"ID":"671255a1-8242-493d-8eb3-760b978433f1","Type":"ContainerDied","Data":"5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc"}
Jul 15 12:18:18.884798 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.884438    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7hjp4" event={"ID":"671255a1-8242-493d-8eb3-760b978433f1","Type":"ContainerDied","Data":"2d6feaca0aafdf52a20f24e616291d0c34ebfb8a561a300a4816e415d3e4644c"}
Jul 15 12:18:18.884798 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.884461    2582 scope.go:117] "RemoveContainer" containerID="5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc"
Jul 15 12:18:18.893968 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.893944    2582 scope.go:117] "RemoveContainer" containerID="5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc"
Jul 15 12:18:18.894316 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:18.894291    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc\": container with ID starting with 5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc not found: ID does not exist" containerID="5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc"
Jul 15 12:18:18.894383 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.894333    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc"} err="failed to get container status \"5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc\": rpc error: code = NotFound desc = could not find container \"5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc\": container with ID starting with 5b86f438d55d51aa8ccb7b9f9e49cca16d0deda1a5e9d556e5aded60e3f362fc not found: ID does not exist"
Jul 15 12:18:18.935380 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.935340    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hjp4"]
Jul 15 12:18:18.944384 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:18.944341    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7hjp4"]
Jul 15 12:18:20.063656 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:20.063619    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="671255a1-8242-493d-8eb3-760b978433f1" path="/var/lib/kubelet/pods/671255a1-8242-493d-8eb3-760b978433f1/volumes"
Jul 15 12:18:23.148021 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.147980    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-697c46b49f-wwklx"]
Jul 15 12:18:23.148578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.148516    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef9e20c-4e41-4722-9eed-f27b5b2af5ac" containerName="authorino"
Jul 15 12:18:23.148578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.148530    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef9e20c-4e41-4722-9eed-f27b5b2af5ac" containerName="authorino"
Jul 15 12:18:23.148578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.148555    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671255a1-8242-493d-8eb3-760b978433f1" containerName="authorino"
Jul 15 12:18:23.148578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.148560    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="671255a1-8242-493d-8eb3-760b978433f1" containerName="authorino"
Jul 15 12:18:23.148726 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.148621    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="671255a1-8242-493d-8eb3-760b978433f1" containerName="authorino"
Jul 15 12:18:23.148726 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.148631    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef9e20c-4e41-4722-9eed-f27b5b2af5ac" containerName="authorino"
Jul 15 12:18:23.163996 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.163963    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:23.166553 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.166517    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 12:18:23.167028 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.167006    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 12:18:23.167230 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.167061    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hz64s\""
Jul 15 12:18:23.168824 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.168796    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-697c46b49f-wwklx"]
Jul 15 12:18:23.331677 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.331632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert\") pod \"maas-controller-697c46b49f-wwklx\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") " pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:23.331867 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.331753    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vngk2\" (UniqueName: \"kubernetes.io/projected/f07b0013-b1d9-40a1-ae57-308dc540fec7-kube-api-access-vngk2\") pod \"maas-controller-697c46b49f-wwklx\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") " pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:23.354692 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.354651    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-289ts"]
Jul 15 12:18:23.357169 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.357142    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:23.369319 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.369289    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-289ts"]
Jul 15 12:18:23.432837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.432741    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert\") pod \"maas-controller-697c46b49f-wwklx\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") " pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:23.432837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.432794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert\") pod \"maas-controller-6f7b86fcb4-289ts\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") " pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:23.432837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.432837    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9rzh\" (UniqueName: \"kubernetes.io/projected/67fa5d37-3dd1-44ca-9501-4ea6fb785933-kube-api-access-l9rzh\") pod \"maas-controller-6f7b86fcb4-289ts\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") " pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:23.433053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.432879    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vngk2\" (UniqueName: \"kubernetes.io/projected/f07b0013-b1d9-40a1-ae57-308dc540fec7-kube-api-access-vngk2\") pod \"maas-controller-697c46b49f-wwklx\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") " pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:23.433053 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:23.432905    2582 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 15 12:18:23.433053 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:23.432988    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert podName:f07b0013-b1d9-40a1-ae57-308dc540fec7 nodeName:}" failed. No retries permitted until 2026-07-15 12:18:23.932968613 +0000 UTC m=+635.032917944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert") pod "maas-controller-697c46b49f-wwklx" (UID: "f07b0013-b1d9-40a1-ae57-308dc540fec7") : secret "maas-controller-webhook-cert" not found
Jul 15 12:18:23.441540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.441514    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vngk2\" (UniqueName: \"kubernetes.io/projected/f07b0013-b1d9-40a1-ae57-308dc540fec7-kube-api-access-vngk2\") pod \"maas-controller-697c46b49f-wwklx\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") " pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:23.533540 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.533492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert\") pod \"maas-controller-6f7b86fcb4-289ts\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") " pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:23.533739 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.533552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9rzh\" (UniqueName: \"kubernetes.io/projected/67fa5d37-3dd1-44ca-9501-4ea6fb785933-kube-api-access-l9rzh\") pod \"maas-controller-6f7b86fcb4-289ts\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") " pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:23.533739 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:23.533644    2582 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 15 12:18:23.533739 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:23.533712    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert podName:67fa5d37-3dd1-44ca-9501-4ea6fb785933 nodeName:}" failed. No retries permitted until 2026-07-15 12:18:24.033693648 +0000 UTC m=+635.133642967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert") pod "maas-controller-6f7b86fcb4-289ts" (UID: "67fa5d37-3dd1-44ca-9501-4ea6fb785933") : secret "maas-controller-webhook-cert" not found
Jul 15 12:18:23.542634 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.542596    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9rzh\" (UniqueName: \"kubernetes.io/projected/67fa5d37-3dd1-44ca-9501-4ea6fb785933-kube-api-access-l9rzh\") pod \"maas-controller-6f7b86fcb4-289ts\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") " pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:23.938191 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.938147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert\") pod \"maas-controller-697c46b49f-wwklx\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") " pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:23.940654 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:23.940631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert\") pod \"maas-controller-697c46b49f-wwklx\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") " pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:24.038740 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.038701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert\") pod \"maas-controller-6f7b86fcb4-289ts\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") " pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:24.041086 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.041050    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert\") pod \"maas-controller-6f7b86fcb4-289ts\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") " pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:24.075929 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.075892    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:24.215307 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.215278    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-697c46b49f-wwklx"]
Jul 15 12:18:24.217382 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:24.217354    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07b0013_b1d9_40a1_ae57_308dc540fec7.slice/crio-6218240e58ed93824138e6c6b13ff0639d4c952960d3b78fc401bcd79f13c32f WatchSource:0}: Error finding container 6218240e58ed93824138e6c6b13ff0639d4c952960d3b78fc401bcd79f13c32f: Status 404 returned error can't find the container with id 6218240e58ed93824138e6c6b13ff0639d4c952960d3b78fc401bcd79f13c32f
Jul 15 12:18:24.267988 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.267945    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:24.410415 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.410386    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-289ts"]
Jul 15 12:18:24.412401 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:24.412373    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fa5d37_3dd1_44ca_9501_4ea6fb785933.slice/crio-437ae168d077d3d956de9be1d8e0e2e8a191cab032c25fa43732ff681f7730d9 WatchSource:0}: Error finding container 437ae168d077d3d956de9be1d8e0e2e8a191cab032c25fa43732ff681f7730d9: Status 404 returned error can't find the container with id 437ae168d077d3d956de9be1d8e0e2e8a191cab032c25fa43732ff681f7730d9
Jul 15 12:18:24.921369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.921328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f7b86fcb4-289ts" event={"ID":"67fa5d37-3dd1-44ca-9501-4ea6fb785933","Type":"ContainerStarted","Data":"437ae168d077d3d956de9be1d8e0e2e8a191cab032c25fa43732ff681f7730d9"}
Jul 15 12:18:24.923124 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:24.923072    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-697c46b49f-wwklx" event={"ID":"f07b0013-b1d9-40a1-ae57-308dc540fec7","Type":"ContainerStarted","Data":"6218240e58ed93824138e6c6b13ff0639d4c952960d3b78fc401bcd79f13c32f"}
Jul 15 12:18:27.946009 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:27.945962    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f7b86fcb4-289ts" event={"ID":"67fa5d37-3dd1-44ca-9501-4ea6fb785933","Type":"ContainerStarted","Data":"7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4"}
Jul 15 12:18:27.946528 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:27.946077    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:27.947538 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:27.947504    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-697c46b49f-wwklx" event={"ID":"f07b0013-b1d9-40a1-ae57-308dc540fec7","Type":"ContainerStarted","Data":"26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc"}
Jul 15 12:18:27.947652 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:27.947637    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:27.968912 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:27.968854    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f7b86fcb4-289ts" podStartSLOduration=2.510968742 podStartE2EDuration="4.968832865s" totalImagesPullingTime="2.457864123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:18:27.964481561 +0000 UTC m=+639.064430901" watchObservedRunningTime="2026-07-15 12:18:27.968832865 +0000 UTC m=+639.068782205"
Jul 15 12:18:27.984248 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:27.984190    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-697c46b49f-wwklx" podStartSLOduration=2.343181525 podStartE2EDuration="4.984175268s" totalImagesPullingTime="2.640993743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:18:27.983152206 +0000 UTC m=+639.083101545" watchObservedRunningTime="2026-07-15 12:18:27.984175268 +0000 UTC m=+639.084124622"
Jul 15 12:18:38.954758 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:38.954722    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:18:38.955152 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:38.954902    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:39.009081 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.009039    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-697c46b49f-wwklx"]
Jul 15 12:18:39.012024 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.011982    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-697c46b49f-wwklx" podUID="f07b0013-b1d9-40a1-ae57-308dc540fec7" containerName="manager" containerID="cri-o://26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc" gracePeriod=30
Jul 15 12:18:39.260913 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.260883    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:39.267272 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.267251    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert\") pod \"f07b0013-b1d9-40a1-ae57-308dc540fec7\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") "
Jul 15 12:18:39.267353 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.267286    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vngk2\" (UniqueName: \"kubernetes.io/projected/f07b0013-b1d9-40a1-ae57-308dc540fec7-kube-api-access-vngk2\") pod \"f07b0013-b1d9-40a1-ae57-308dc540fec7\" (UID: \"f07b0013-b1d9-40a1-ae57-308dc540fec7\") "
Jul 15 12:18:39.269378 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.269352    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f07b0013-b1d9-40a1-ae57-308dc540fec7" (UID: "f07b0013-b1d9-40a1-ae57-308dc540fec7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:18:39.269480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.269459    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f07b0013-b1d9-40a1-ae57-308dc540fec7-kube-api-access-vngk2" (OuterVolumeSpecName: "kube-api-access-vngk2") pod "f07b0013-b1d9-40a1-ae57-308dc540fec7" (UID: "f07b0013-b1d9-40a1-ae57-308dc540fec7"). InnerVolumeSpecName "kube-api-access-vngk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:18:39.368223 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.368189    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f07b0013-b1d9-40a1-ae57-308dc540fec7-webhook-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:18:39.368223 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:39.368220    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vngk2\" (UniqueName: \"kubernetes.io/projected/f07b0013-b1d9-40a1-ae57-308dc540fec7-kube-api-access-vngk2\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:18:40.018597 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.018559    2582 generic.go:356] "Generic (PLEG): container finished" podID="f07b0013-b1d9-40a1-ae57-308dc540fec7" containerID="26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc" exitCode=0
Jul 15 12:18:40.019052 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.018605    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-697c46b49f-wwklx"
Jul 15 12:18:40.019052 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.018643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-697c46b49f-wwklx" event={"ID":"f07b0013-b1d9-40a1-ae57-308dc540fec7","Type":"ContainerDied","Data":"26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc"}
Jul 15 12:18:40.019052 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.018699    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-697c46b49f-wwklx" event={"ID":"f07b0013-b1d9-40a1-ae57-308dc540fec7","Type":"ContainerDied","Data":"6218240e58ed93824138e6c6b13ff0639d4c952960d3b78fc401bcd79f13c32f"}
Jul 15 12:18:40.019052 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.018716    2582 scope.go:117] "RemoveContainer" containerID="26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc"
Jul 15 12:18:40.028221 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.028203    2582 scope.go:117] "RemoveContainer" containerID="26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc"
Jul 15 12:18:40.028528 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:18:40.028509    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc\": container with ID starting with 26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc not found: ID does not exist" containerID="26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc"
Jul 15 12:18:40.028575 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.028539    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc"} err="failed to get container status \"26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc\": rpc error: code = NotFound desc = could not find container \"26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc\": container with ID starting with 26c35d513a6580e37f410465ff1c217207855bca9cbc4f79da0c12c78a8edfdc not found: ID does not exist"
Jul 15 12:18:40.042590 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.042556    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-697c46b49f-wwklx"]
Jul 15 12:18:40.046542 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.046516    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-697c46b49f-wwklx"]
Jul 15 12:18:40.062276 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:40.062239    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f07b0013-b1d9-40a1-ae57-308dc540fec7" path="/var/lib/kubelet/pods/f07b0013-b1d9-40a1-ae57-308dc540fec7/volumes"
Jul 15 12:18:57.593738 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.593702    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"]
Jul 15 12:18:57.594281 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.594153    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f07b0013-b1d9-40a1-ae57-308dc540fec7" containerName="manager"
Jul 15 12:18:57.594281 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.594172    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f07b0013-b1d9-40a1-ae57-308dc540fec7" containerName="manager"
Jul 15 12:18:57.594281 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.594274    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f07b0013-b1d9-40a1-ae57-308dc540fec7" containerName="manager"
Jul 15 12:18:57.598510 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.598494    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.600404 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.600378    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 12:18:57.609354 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.609330    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"]
Jul 15 12:18:57.716887 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.716845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/624f60ca-5e9e-4882-a658-833af66d83be-maas-api-tls\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.716887 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.716888    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/624f60ca-5e9e-4882-a658-833af66d83be-openshift-service-ca\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.717242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.717004    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwpf5\" (UniqueName: \"kubernetes.io/projected/624f60ca-5e9e-4882-a658-833af66d83be-kube-api-access-cwpf5\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.818325 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.818290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwpf5\" (UniqueName: \"kubernetes.io/projected/624f60ca-5e9e-4882-a658-833af66d83be-kube-api-access-cwpf5\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.818497 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.818351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/624f60ca-5e9e-4882-a658-833af66d83be-maas-api-tls\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.818497 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.818461    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/624f60ca-5e9e-4882-a658-833af66d83be-openshift-service-ca\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.819043 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.819022    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/624f60ca-5e9e-4882-a658-833af66d83be-openshift-service-ca\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.820712 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.820684    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/624f60ca-5e9e-4882-a658-833af66d83be-maas-api-tls\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.826840 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.826813    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwpf5\" (UniqueName: \"kubernetes.io/projected/624f60ca-5e9e-4882-a658-833af66d83be-kube-api-access-cwpf5\") pod \"maas-api-5bd77b5849-wcbf2\" (UID: \"624f60ca-5e9e-4882-a658-833af66d83be\") " pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:57.909773 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:57.909667    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:18:58.062809 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:58.062773    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624f60ca_5e9e_4882_a658_833af66d83be.slice/crio-5626d91a2e75e32d0b12f290c5ec079f009bdb2d37efb1b5d3523a4e18cf1fb6 WatchSource:0}: Error finding container 5626d91a2e75e32d0b12f290c5ec079f009bdb2d37efb1b5d3523a4e18cf1fb6: Status 404 returned error can't find the container with id 5626d91a2e75e32d0b12f290c5ec079f009bdb2d37efb1b5d3523a4e18cf1fb6
Jul 15 12:18:58.063416 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.063286    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"]
Jul 15 12:18:58.063993 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.063976    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:18:58.129035 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.128997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2" event={"ID":"624f60ca-5e9e-4882-a658-833af66d83be","Type":"ContainerStarted","Data":"5626d91a2e75e32d0b12f290c5ec079f009bdb2d37efb1b5d3523a4e18cf1fb6"}
Jul 15 12:18:58.286389 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.286348    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-b27b7"]
Jul 15 12:18:58.291070 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.291048    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.293036 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.293016    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 12:18:58.293176 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.293053    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rhpqh\""
Jul 15 12:18:58.300446 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.300420    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-b27b7"]
Jul 15 12:18:58.320948 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.320913    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"]
Jul 15 12:18:58.321296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.321275    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac44c916-a9bf-42e8-9db6-1276498c9fb3-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-b27b7\" (UID: \"ac44c916-a9bf-42e8-9db6-1276498c9fb3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.321349 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.321305    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crgp6\" (UniqueName: \"kubernetes.io/projected/ac44c916-a9bf-42e8-9db6-1276498c9fb3-kube-api-access-crgp6\") pod \"payload-processing-f9b8d9b9b-b27b7\" (UID: \"ac44c916-a9bf-42e8-9db6-1276498c9fb3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.324463 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.324448    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.334033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.334008    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"]
Jul 15 12:18:58.422734 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.422695    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95x74\" (UniqueName: \"kubernetes.io/projected/dcd2520a-433d-457a-9e73-8f983b7ef53d-kube-api-access-95x74\") pod \"payload-pre-processing-74584bb9f8-ldb5r\" (UID: \"dcd2520a-433d-457a-9e73-8f983b7ef53d\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.422967 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.422748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcd2520a-433d-457a-9e73-8f983b7ef53d-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-ldb5r\" (UID: \"dcd2520a-433d-457a-9e73-8f983b7ef53d\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.422967 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.422852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac44c916-a9bf-42e8-9db6-1276498c9fb3-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-b27b7\" (UID: \"ac44c916-a9bf-42e8-9db6-1276498c9fb3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.422967 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.422889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crgp6\" (UniqueName: \"kubernetes.io/projected/ac44c916-a9bf-42e8-9db6-1276498c9fb3-kube-api-access-crgp6\") pod \"payload-processing-f9b8d9b9b-b27b7\" (UID: \"ac44c916-a9bf-42e8-9db6-1276498c9fb3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.423612 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.423592    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac44c916-a9bf-42e8-9db6-1276498c9fb3-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-b27b7\" (UID: \"ac44c916-a9bf-42e8-9db6-1276498c9fb3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.431717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.431689    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crgp6\" (UniqueName: \"kubernetes.io/projected/ac44c916-a9bf-42e8-9db6-1276498c9fb3-kube-api-access-crgp6\") pod \"payload-processing-f9b8d9b9b-b27b7\" (UID: \"ac44c916-a9bf-42e8-9db6-1276498c9fb3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.523664 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.523618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95x74\" (UniqueName: \"kubernetes.io/projected/dcd2520a-433d-457a-9e73-8f983b7ef53d-kube-api-access-95x74\") pod \"payload-pre-processing-74584bb9f8-ldb5r\" (UID: \"dcd2520a-433d-457a-9e73-8f983b7ef53d\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.523858 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.523683    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcd2520a-433d-457a-9e73-8f983b7ef53d-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-ldb5r\" (UID: \"dcd2520a-433d-457a-9e73-8f983b7ef53d\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.524450 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.524428    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcd2520a-433d-457a-9e73-8f983b7ef53d-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-ldb5r\" (UID: \"dcd2520a-433d-457a-9e73-8f983b7ef53d\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.538024 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.537934    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95x74\" (UniqueName: \"kubernetes.io/projected/dcd2520a-433d-457a-9e73-8f983b7ef53d-kube-api-access-95x74\") pod \"payload-pre-processing-74584bb9f8-ldb5r\" (UID: \"dcd2520a-433d-457a-9e73-8f983b7ef53d\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.602750 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.602712    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:18:58.636040 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.636003    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:18:58.776185 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.776029    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-b27b7"]
Jul 15 12:18:58.778803 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:58.778770    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac44c916_a9bf_42e8_9db6_1276498c9fb3.slice/crio-2fb10546f0dafaac6eda6983f895c53674291f63122f9b39d23b1543b8c3dfd8 WatchSource:0}: Error finding container 2fb10546f0dafaac6eda6983f895c53674291f63122f9b39d23b1543b8c3dfd8: Status 404 returned error can't find the container with id 2fb10546f0dafaac6eda6983f895c53674291f63122f9b39d23b1543b8c3dfd8
Jul 15 12:18:58.804797 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:58.804763    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"]
Jul 15 12:18:58.806330 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:18:58.806285    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd2520a_433d_457a_9e73_8f983b7ef53d.slice/crio-a019d861e24ac0265ab7a82d380dcb9e7fce5ae51dcaf63ac3cb0ae14371fe4d WatchSource:0}: Error finding container a019d861e24ac0265ab7a82d380dcb9e7fce5ae51dcaf63ac3cb0ae14371fe4d: Status 404 returned error can't find the container with id a019d861e24ac0265ab7a82d380dcb9e7fce5ae51dcaf63ac3cb0ae14371fe4d
Jul 15 12:18:59.139467 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:59.139337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r" event={"ID":"dcd2520a-433d-457a-9e73-8f983b7ef53d","Type":"ContainerStarted","Data":"a019d861e24ac0265ab7a82d380dcb9e7fce5ae51dcaf63ac3cb0ae14371fe4d"}
Jul 15 12:18:59.140997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:18:59.140963    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7" event={"ID":"ac44c916-a9bf-42e8-9db6-1276498c9fb3","Type":"ContainerStarted","Data":"2fb10546f0dafaac6eda6983f895c53674291f63122f9b39d23b1543b8c3dfd8"}
Jul 15 12:19:01.158685 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.158648    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7" event={"ID":"ac44c916-a9bf-42e8-9db6-1276498c9fb3","Type":"ContainerStarted","Data":"eb4590f1de219f9c9900437d1be7eee8cf296259ad2b0a4db1e78827b0774183"}
Jul 15 12:19:01.159143 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.158798    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:19:01.160269 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.160241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r" event={"ID":"dcd2520a-433d-457a-9e73-8f983b7ef53d","Type":"ContainerStarted","Data":"c5ffb370605a006bb880286bcf963d8634c8cbdabd10bc8b66f5f8da4601a545"}
Jul 15 12:19:01.160402 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.160291    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:19:01.161717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.161697    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2" event={"ID":"624f60ca-5e9e-4882-a658-833af66d83be","Type":"ContainerStarted","Data":"08d4bd5308947fe9604719739fcdec6bf6353d6a67021008d9e5b70a37ad1e1f"}
Jul 15 12:19:01.161809 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.161752    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:19:01.178391 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.178340    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7" podStartSLOduration=1.6862077100000001 podStartE2EDuration="3.17832434s" totalImagesPullingTime="1.49211663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:19:01.176233811 +0000 UTC m=+672.276183149" watchObservedRunningTime="2026-07-15 12:19:01.17832434 +0000 UTC m=+672.278273678"
Jul 15 12:19:01.190920 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.190866    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r" podStartSLOduration=1.728343562 podStartE2EDuration="3.190848243s" totalImagesPullingTime="1.462504681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:19:01.190193392 +0000 UTC m=+672.290142753" watchObservedRunningTime="2026-07-15 12:19:01.190848243 +0000 UTC m=+672.290797634"
Jul 15 12:19:01.206535 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:01.206481    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2" podStartSLOduration=1.999632694 podStartE2EDuration="4.206463485s" totalImagesPullingTime="2.206830791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:19:01.205527296 +0000 UTC m=+672.305476669" watchObservedRunningTime="2026-07-15 12:19:01.206463485 +0000 UTC m=+672.306412824"
Jul 15 12:19:07.172225 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:07.172194    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5bd77b5849-wcbf2"
Jul 15 12:19:12.168941 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:12.168908    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-b27b7"
Jul 15 12:19:12.169452 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:12.169252    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-ldb5r"
Jul 15 12:19:45.720257 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.720175    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"]
Jul 15 12:19:45.722755 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.722730    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.727727 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.727695    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7rfd5\""
Jul 15 12:19:45.727727 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.727714    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 12:19:45.727922 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.727734    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 15 12:19:45.727922 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.727710    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 12:19:45.728319 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.728301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.728374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.728328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.728374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.728366    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwdwd\" (UniqueName: \"kubernetes.io/projected/b0de936b-ebf2-4498-97e7-e1fcc765b091-kube-api-access-lwdwd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.728450 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.728429    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.728496 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.728487    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0de936b-ebf2-4498-97e7-e1fcc765b091-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.728534 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.728518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.743868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.743836    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"]
Jul 15 12:19:45.828937 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.828895    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwdwd\" (UniqueName: \"kubernetes.io/projected/b0de936b-ebf2-4498-97e7-e1fcc765b091-kube-api-access-lwdwd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829165 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.828951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829165 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.829005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0de936b-ebf2-4498-97e7-e1fcc765b091-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829165 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.829041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829165 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.829067    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829165 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.829122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829484 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.829460    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.829472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.829548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.829520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.831569 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.831541    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0de936b-ebf2-4498-97e7-e1fcc765b091-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.831828 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.831806    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0de936b-ebf2-4498-97e7-e1fcc765b091-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:45.843349 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:45.843315    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwdwd\" (UniqueName: \"kubernetes.io/projected/b0de936b-ebf2-4498-97e7-e1fcc765b091-kube-api-access-lwdwd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7\" (UID: \"b0de936b-ebf2-4498-97e7-e1fcc765b091\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:46.033810 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:46.033706    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:46.183832 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:46.183802    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"]
Jul 15 12:19:46.185399 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:19:46.185375    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0de936b_ebf2_4498_97e7_e1fcc765b091.slice/crio-e8a1de404a1aa57b55c2f58f98b473aaaf6b7d726317fa88b4da27f053d54478 WatchSource:0}: Error finding container e8a1de404a1aa57b55c2f58f98b473aaaf6b7d726317fa88b4da27f053d54478: Status 404 returned error can't find the container with id e8a1de404a1aa57b55c2f58f98b473aaaf6b7d726317fa88b4da27f053d54478
Jul 15 12:19:46.402319 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:46.402231    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7" event={"ID":"b0de936b-ebf2-4498-97e7-e1fcc765b091","Type":"ContainerStarted","Data":"e8a1de404a1aa57b55c2f58f98b473aaaf6b7d726317fa88b4da27f053d54478"}
Jul 15 12:19:48.416280 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:48.416245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7" event={"ID":"b0de936b-ebf2-4498-97e7-e1fcc765b091","Type":"ContainerStarted","Data":"26a380629c05df7e6e897ad5b78b31667cc3db05233e6a6076241e8adae819bd"}
Jul 15 12:19:48.439384 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:48.439320    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7" podStartSLOduration=2.162359268 podStartE2EDuration="3.439303617s" totalImagesPullingTime="1.276944349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:19:48.436734348 +0000 UTC m=+719.536683693" watchObservedRunningTime="2026-07-15 12:19:48.439303617 +0000 UTC m=+719.539252956"
Jul 15 12:19:51.894264 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.894223    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"]
Jul 15 12:19:51.897374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.897347    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:51.899510 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.899483    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 15 12:19:51.906118 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.906079    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"]
Jul 15 12:19:51.974650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.974598    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:51.974846 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.974662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vgvb\" (UniqueName: \"kubernetes.io/projected/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-kube-api-access-2vgvb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:51.974846 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.974726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:51.974846 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.974796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:51.974846 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.974825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:51.974846 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:51.974846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075266 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vgvb\" (UniqueName: \"kubernetes.io/projected/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-kube-api-access-2vgvb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075480 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075805 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075780    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075805 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075796    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.075918 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.075848    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.077537 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.077519    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.077792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.077774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.083521 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.083493    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vgvb\" (UniqueName: \"kubernetes.io/projected/d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2-kube-api-access-2vgvb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wj9jv\" (UID: \"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.210194 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.210163    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:19:52.352458 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.352429    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"]
Jul 15 12:19:52.353912 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:19:52.353883    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd720a5ec_1c36_45bb_86e4_a10fb4ebbfd2.slice/crio-1bc51f340ed95314cf2752db80ecf164fcbed686cf88fad8e26ea060606bd48f WatchSource:0}: Error finding container 1bc51f340ed95314cf2752db80ecf164fcbed686cf88fad8e26ea060606bd48f: Status 404 returned error can't find the container with id 1bc51f340ed95314cf2752db80ecf164fcbed686cf88fad8e26ea060606bd48f
Jul 15 12:19:52.439720 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:52.439687    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv" event={"ID":"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2","Type":"ContainerStarted","Data":"1bc51f340ed95314cf2752db80ecf164fcbed686cf88fad8e26ea060606bd48f"}
Jul 15 12:19:53.449403 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:53.449360    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv" event={"ID":"d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2","Type":"ContainerStarted","Data":"bf1745b2d8c9399b6996025d35ab5ea1c1c4aeb74e3bfdb8629c5290e2f7f089"}
Jul 15 12:19:53.472564 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:53.472499    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv" podStartSLOduration=2.297586382 podStartE2EDuration="2.472482702s" totalImagesPullingTime="174.89632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:19:53.470118224 +0000 UTC m=+724.570067557" watchObservedRunningTime="2026-07-15 12:19:53.472482702 +0000 UTC m=+724.572432041"
Jul 15 12:19:56.034294 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:56.034240    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:56.034709 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:56.034325    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:56.046874 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:56.046843    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:19:56.479849 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:19:56.479816    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7"
Jul 15 12:20:02.210889 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:02.210845    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:20:02.211325 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:02.210913    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:20:02.223618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:02.223591    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:20:02.513824 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:02.513729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wj9jv"
Jul 15 12:20:31.400994 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.400962    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"]
Jul 15 12:20:31.403571 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.403549    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.406033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.406013    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 15 12:20:31.416081 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.416059    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"]
Jul 15 12:20:31.516373 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.516332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16a01935-c104-45d6-bece-c23ab671c835-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.516373 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.516378    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.516606 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.516400    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zcfw\" (UniqueName: \"kubernetes.io/projected/16a01935-c104-45d6-bece-c23ab671c835-kube-api-access-2zcfw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.516606 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.516501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.516606 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.516557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.516606 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.516577    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.617497 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.617497 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.617751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16a01935-c104-45d6-bece-c23ab671c835-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.617751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617556    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.617751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zcfw\" (UniqueName: \"kubernetes.io/projected/16a01935-c104-45d6-bece-c23ab671c835-kube-api-access-2zcfw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.617751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.617958 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617919    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.618011 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617951    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.618011 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.617959    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.619940 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.619910    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16a01935-c104-45d6-bece-c23ab671c835-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.620061 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.620044    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16a01935-c104-45d6-bece-c23ab671c835-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.626633 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.626606    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zcfw\" (UniqueName: \"kubernetes.io/projected/16a01935-c104-45d6-bece-c23ab671c835-kube-api-access-2zcfw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7\" (UID: \"16a01935-c104-45d6-bece-c23ab671c835\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.714472 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.714424    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:31.868038 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:31.868008    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"]
Jul 15 12:20:31.869938 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:20:31.869911    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a01935_c104_45d6_bece_c23ab671c835.slice/crio-1368d194d420aa2bb5f1cc4ce0d399a2fb7d737df76e77a0bb7d82dd0aa9826c WatchSource:0}: Error finding container 1368d194d420aa2bb5f1cc4ce0d399a2fb7d737df76e77a0bb7d82dd0aa9826c: Status 404 returned error can't find the container with id 1368d194d420aa2bb5f1cc4ce0d399a2fb7d737df76e77a0bb7d82dd0aa9826c
Jul 15 12:20:32.685989 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:32.685951    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7" event={"ID":"16a01935-c104-45d6-bece-c23ab671c835","Type":"ContainerStarted","Data":"2796223a2ff02f860ea83ba73403c67c6a5135aaaa9824134d0694c1fc18ccea"}
Jul 15 12:20:32.686418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:32.686005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7" event={"ID":"16a01935-c104-45d6-bece-c23ab671c835","Type":"ContainerStarted","Data":"1368d194d420aa2bb5f1cc4ce0d399a2fb7d737df76e77a0bb7d82dd0aa9826c"}
Jul 15 12:20:32.712047 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:32.711991    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7" podStartSLOduration=1.530754822 podStartE2EDuration="1.711973688s" totalImagesPullingTime="181.218866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:32.70888439 +0000 UTC m=+763.808833731" watchObservedRunningTime="2026-07-15 12:20:32.711973688 +0000 UTC m=+763.811923026"
Jul 15 12:20:41.714876 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:41.714826    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:41.715375 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:41.714913    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:41.727956 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:41.727920    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:41.747462 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:41.747431    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7"
Jul 15 12:20:44.730002 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.729964    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"]
Jul 15 12:20:44.733520 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.733494    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.736574 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.736551    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 15 12:20:44.799348 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.799318    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"]
Jul 15 12:20:44.841045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.841003    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.841247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.841060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76c783fd-c346-45c2-8039-6f3dd23514c9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.841247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.841133    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.841247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.841178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.841247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.841221    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9bqb\" (UniqueName: \"kubernetes.io/projected/76c783fd-c346-45c2-8039-6f3dd23514c9-kube-api-access-b9bqb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.841448 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.841321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.942664 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.942627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9bqb\" (UniqueName: \"kubernetes.io/projected/76c783fd-c346-45c2-8039-6f3dd23514c9-kube-api-access-b9bqb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.942858 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.942684    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.942858 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.942743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.942858 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.942773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76c783fd-c346-45c2-8039-6f3dd23514c9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.942858 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.942808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.942858 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.942854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.943225 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.943200    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.943297 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.943230    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.943344 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.943323    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.945079 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.945053    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76c783fd-c346-45c2-8039-6f3dd23514c9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.945265 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.945250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76c783fd-c346-45c2-8039-6f3dd23514c9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:44.953064 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:44.953040    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9bqb\" (UniqueName: \"kubernetes.io/projected/76c783fd-c346-45c2-8039-6f3dd23514c9-kube-api-access-b9bqb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp\" (UID: \"76c783fd-c346-45c2-8039-6f3dd23514c9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:45.043128 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:45.043000    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:45.211077 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:45.211049    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"]
Jul 15 12:20:45.213411 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:20:45.213376    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76c783fd_c346_45c2_8039_6f3dd23514c9.slice/crio-c59f11c817107b8de940657633d0ee5fe8298b6bd84eff6816776952da79ca34 WatchSource:0}: Error finding container c59f11c817107b8de940657633d0ee5fe8298b6bd84eff6816776952da79ca34: Status 404 returned error can't find the container with id c59f11c817107b8de940657633d0ee5fe8298b6bd84eff6816776952da79ca34
Jul 15 12:20:45.761801 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:45.761734    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp" event={"ID":"76c783fd-c346-45c2-8039-6f3dd23514c9","Type":"ContainerStarted","Data":"80c72f74e9f6e17a7d14bdbd6fb739c5dff45383723ef84ad31ebe0efc17753d"}
Jul 15 12:20:45.762252 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:45.761815    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp" event={"ID":"76c783fd-c346-45c2-8039-6f3dd23514c9","Type":"ContainerStarted","Data":"c59f11c817107b8de940657633d0ee5fe8298b6bd84eff6816776952da79ca34"}
Jul 15 12:20:45.782001 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:45.781949    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp" podStartSLOduration=1.558480634 podStartE2EDuration="1.781929576s" totalImagesPullingTime="223.448942ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:45.781138146 +0000 UTC m=+776.881087486" watchObservedRunningTime="2026-07-15 12:20:45.781929576 +0000 UTC m=+776.881878916"
Jul 15 12:20:55.043214 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:55.043174    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:55.043719 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:55.043234    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:55.056488 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:55.056453    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:20:55.833890 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:20:55.833851    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp"
Jul 15 12:21:49.018621 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.018580    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68b5dccdcc-pvzbl"]
Jul 15 12:21:49.022211 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.022190    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.031202 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.031173    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68b5dccdcc-pvzbl"]
Jul 15 12:21:49.112575 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.112534    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk4nj\" (UniqueName: \"kubernetes.io/projected/2c53bec2-a7a0-49c8-9062-c26dd7aafad0-kube-api-access-qk4nj\") pod \"authorino-68b5dccdcc-pvzbl\" (UID: \"2c53bec2-a7a0-49c8-9062-c26dd7aafad0\") " pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.112752 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.112589    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c53bec2-a7a0-49c8-9062-c26dd7aafad0-tls-cert\") pod \"authorino-68b5dccdcc-pvzbl\" (UID: \"2c53bec2-a7a0-49c8-9062-c26dd7aafad0\") " pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.213180 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.213145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk4nj\" (UniqueName: \"kubernetes.io/projected/2c53bec2-a7a0-49c8-9062-c26dd7aafad0-kube-api-access-qk4nj\") pod \"authorino-68b5dccdcc-pvzbl\" (UID: \"2c53bec2-a7a0-49c8-9062-c26dd7aafad0\") " pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.213339 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.213191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c53bec2-a7a0-49c8-9062-c26dd7aafad0-tls-cert\") pod \"authorino-68b5dccdcc-pvzbl\" (UID: \"2c53bec2-a7a0-49c8-9062-c26dd7aafad0\") " pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.215570 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.215542    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c53bec2-a7a0-49c8-9062-c26dd7aafad0-tls-cert\") pod \"authorino-68b5dccdcc-pvzbl\" (UID: \"2c53bec2-a7a0-49c8-9062-c26dd7aafad0\") " pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.223691 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.223668    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk4nj\" (UniqueName: \"kubernetes.io/projected/2c53bec2-a7a0-49c8-9062-c26dd7aafad0-kube-api-access-qk4nj\") pod \"authorino-68b5dccdcc-pvzbl\" (UID: \"2c53bec2-a7a0-49c8-9062-c26dd7aafad0\") " pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.332166 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.332058    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68b5dccdcc-pvzbl"
Jul 15 12:21:49.492034 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:49.492007    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68b5dccdcc-pvzbl"]
Jul 15 12:21:49.494358 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:21:49.494331    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c53bec2_a7a0_49c8_9062_c26dd7aafad0.slice/crio-32b5876c421e27d096221e243979f2e071bf44ff8bcd53d3e19e51306831e55f WatchSource:0}: Error finding container 32b5876c421e27d096221e243979f2e071bf44ff8bcd53d3e19e51306831e55f: Status 404 returned error can't find the container with id 32b5876c421e27d096221e243979f2e071bf44ff8bcd53d3e19e51306831e55f
Jul 15 12:21:50.133145 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.133078    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68b5dccdcc-pvzbl" event={"ID":"2c53bec2-a7a0-49c8-9062-c26dd7aafad0","Type":"ContainerStarted","Data":"8ab6870015bbdb490502d8515f8f2eca3489f6b3baa59ffc2688787d48d2abe5"}
Jul 15 12:21:50.133740 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.133698    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68b5dccdcc-pvzbl" event={"ID":"2c53bec2-a7a0-49c8-9062-c26dd7aafad0","Type":"ContainerStarted","Data":"32b5876c421e27d096221e243979f2e071bf44ff8bcd53d3e19e51306831e55f"}
Jul 15 12:21:50.152981 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.152916    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-68b5dccdcc-pvzbl" podStartSLOduration=0.601689383 podStartE2EDuration="1.152893301s" totalImagesPullingTime="551.203918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:50.151473553 +0000 UTC m=+841.251422903" watchObservedRunningTime="2026-07-15 12:21:50.152893301 +0000 UTC m=+841.252842641"
Jul 15 12:21:50.194300 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.194204    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cc48ff444-pxspr"]
Jul 15 12:21:50.194524 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.194425    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cc48ff444-pxspr" podUID="0d4ac582-d212-4342-bebc-b3afc5f20bd7" containerName="authorino" containerID="cri-o://4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e" gracePeriod=30
Jul 15 12:21:50.465284 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.465255    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:21:50.526311 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.526271    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d4ac582-d212-4342-bebc-b3afc5f20bd7-tls-cert\") pod \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") "
Jul 15 12:21:50.526491 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.526360    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s4zv\" (UniqueName: \"kubernetes.io/projected/0d4ac582-d212-4342-bebc-b3afc5f20bd7-kube-api-access-4s4zv\") pod \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\" (UID: \"0d4ac582-d212-4342-bebc-b3afc5f20bd7\") "
Jul 15 12:21:50.528625 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.528590    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4ac582-d212-4342-bebc-b3afc5f20bd7-kube-api-access-4s4zv" (OuterVolumeSpecName: "kube-api-access-4s4zv") pod "0d4ac582-d212-4342-bebc-b3afc5f20bd7" (UID: "0d4ac582-d212-4342-bebc-b3afc5f20bd7"). InnerVolumeSpecName "kube-api-access-4s4zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:21:50.538475 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.538442    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d4ac582-d212-4342-bebc-b3afc5f20bd7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0d4ac582-d212-4342-bebc-b3afc5f20bd7" (UID: "0d4ac582-d212-4342-bebc-b3afc5f20bd7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:21:50.627972 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.627885    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d4ac582-d212-4342-bebc-b3afc5f20bd7-tls-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:21:50.627972 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:50.627918    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s4zv\" (UniqueName: \"kubernetes.io/projected/0d4ac582-d212-4342-bebc-b3afc5f20bd7-kube-api-access-4s4zv\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:21:51.142899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.142862    2582 generic.go:356] "Generic (PLEG): container finished" podID="0d4ac582-d212-4342-bebc-b3afc5f20bd7" containerID="4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e" exitCode=0
Jul 15 12:21:51.143439 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.142905    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc48ff444-pxspr"
Jul 15 12:21:51.143439 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.142947    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc48ff444-pxspr" event={"ID":"0d4ac582-d212-4342-bebc-b3afc5f20bd7","Type":"ContainerDied","Data":"4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e"}
Jul 15 12:21:51.143439 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.142992    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc48ff444-pxspr" event={"ID":"0d4ac582-d212-4342-bebc-b3afc5f20bd7","Type":"ContainerDied","Data":"ee2975a8735ea5686828d4b0738b6aff295c3b88bc4d0dece696ef31ec28e6b0"}
Jul 15 12:21:51.143439 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.143016    2582 scope.go:117] "RemoveContainer" containerID="4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e"
Jul 15 12:21:51.153710 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.153677    2582 scope.go:117] "RemoveContainer" containerID="4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e"
Jul 15 12:21:51.154043 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:21:51.154022    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e\": container with ID starting with 4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e not found: ID does not exist" containerID="4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e"
Jul 15 12:21:51.154112 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.154053    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e"} err="failed to get container status \"4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e\": rpc error: code = NotFound desc = could not find container \"4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e\": container with ID starting with 4b6cd64cba70883d96103c9ef0fdeb6d5a5fd70eaeab4d1124cd3064f0aa0e0e not found: ID does not exist"
Jul 15 12:21:51.175216 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.175182    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cc48ff444-pxspr"]
Jul 15 12:21:51.181714 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:51.181678    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cc48ff444-pxspr"]
Jul 15 12:21:52.064783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:21:52.064737    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d4ac582-d212-4342-bebc-b3afc5f20bd7" path="/var/lib/kubelet/pods/0d4ac582-d212-4342-bebc-b3afc5f20bd7/volumes"
Jul 15 12:22:49.415204 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:22:49.415110    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:22:49.416784 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:22:49.416758    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:23:01.896140 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:01.896064    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-289ts"]
Jul 15 12:23:01.896633 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:01.896349    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f7b86fcb4-289ts" podUID="67fa5d37-3dd1-44ca-9501-4ea6fb785933" containerName="manager" containerID="cri-o://7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4" gracePeriod=30
Jul 15 12:23:02.160444 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.160372    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:23:02.292440 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.292405    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9rzh\" (UniqueName: \"kubernetes.io/projected/67fa5d37-3dd1-44ca-9501-4ea6fb785933-kube-api-access-l9rzh\") pod \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") "
Jul 15 12:23:02.292627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.292470    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert\") pod \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\" (UID: \"67fa5d37-3dd1-44ca-9501-4ea6fb785933\") "
Jul 15 12:23:02.294569 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.294530    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "67fa5d37-3dd1-44ca-9501-4ea6fb785933" (UID: "67fa5d37-3dd1-44ca-9501-4ea6fb785933"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:23:02.294670 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.294583    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67fa5d37-3dd1-44ca-9501-4ea6fb785933-kube-api-access-l9rzh" (OuterVolumeSpecName: "kube-api-access-l9rzh") pod "67fa5d37-3dd1-44ca-9501-4ea6fb785933" (UID: "67fa5d37-3dd1-44ca-9501-4ea6fb785933"). InnerVolumeSpecName "kube-api-access-l9rzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:23:02.393254 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.393219    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9rzh\" (UniqueName: \"kubernetes.io/projected/67fa5d37-3dd1-44ca-9501-4ea6fb785933-kube-api-access-l9rzh\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:23:02.393254 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.393255    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67fa5d37-3dd1-44ca-9501-4ea6fb785933-webhook-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:23:02.554677 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.554637    2582 generic.go:356] "Generic (PLEG): container finished" podID="67fa5d37-3dd1-44ca-9501-4ea6fb785933" containerID="7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4" exitCode=0
Jul 15 12:23:02.554677 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.554682    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f7b86fcb4-289ts"
Jul 15 12:23:02.554947 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.554726    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f7b86fcb4-289ts" event={"ID":"67fa5d37-3dd1-44ca-9501-4ea6fb785933","Type":"ContainerDied","Data":"7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4"}
Jul 15 12:23:02.554947 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.554781    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f7b86fcb4-289ts" event={"ID":"67fa5d37-3dd1-44ca-9501-4ea6fb785933","Type":"ContainerDied","Data":"437ae168d077d3d956de9be1d8e0e2e8a191cab032c25fa43732ff681f7730d9"}
Jul 15 12:23:02.554947 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.554813    2582 scope.go:117] "RemoveContainer" containerID="7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4"
Jul 15 12:23:02.564139 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.564119    2582 scope.go:117] "RemoveContainer" containerID="7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4"
Jul 15 12:23:02.564385 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:23:02.564368    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4\": container with ID starting with 7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4 not found: ID does not exist" containerID="7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4"
Jul 15 12:23:02.564439 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.564394    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4"} err="failed to get container status \"7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4\": rpc error: code = NotFound desc = could not find container \"7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4\": container with ID starting with 7cb7c2ff29c66f91ce558a226c2b7bea662e57832fb6773c8f78e39dc93158c4 not found: ID does not exist"
Jul 15 12:23:02.579484 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.579451    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-289ts"]
Jul 15 12:23:02.582115 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:02.582073    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-289ts"]
Jul 15 12:23:03.364598 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.364559    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-z7zsn"]
Jul 15 12:23:03.364993 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.364945    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67fa5d37-3dd1-44ca-9501-4ea6fb785933" containerName="manager"
Jul 15 12:23:03.364993 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.364955    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="67fa5d37-3dd1-44ca-9501-4ea6fb785933" containerName="manager"
Jul 15 12:23:03.364993 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.364973    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4ac582-d212-4342-bebc-b3afc5f20bd7" containerName="authorino"
Jul 15 12:23:03.364993 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.364979    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4ac582-d212-4342-bebc-b3afc5f20bd7" containerName="authorino"
Jul 15 12:23:03.365162 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.365054    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4ac582-d212-4342-bebc-b3afc5f20bd7" containerName="authorino"
Jul 15 12:23:03.365162 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.365065    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="67fa5d37-3dd1-44ca-9501-4ea6fb785933" containerName="manager"
Jul 15 12:23:03.369447 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.369428    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.374358 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.374338    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 12:23:03.374485 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.374384    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 12:23:03.374485 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.374340    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hz64s\""
Jul 15 12:23:03.380358 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.380334    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-z7zsn"]
Jul 15 12:23:03.502735 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.502696    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxnd7\" (UniqueName: \"kubernetes.io/projected/011f2896-4654-4374-8bab-cc76ec767193-kube-api-access-kxnd7\") pod \"maas-controller-6f7b86fcb4-z7zsn\" (UID: \"011f2896-4654-4374-8bab-cc76ec767193\") " pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.502911 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.502763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/011f2896-4654-4374-8bab-cc76ec767193-webhook-cert\") pod \"maas-controller-6f7b86fcb4-z7zsn\" (UID: \"011f2896-4654-4374-8bab-cc76ec767193\") " pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.603719 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.603675    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxnd7\" (UniqueName: \"kubernetes.io/projected/011f2896-4654-4374-8bab-cc76ec767193-kube-api-access-kxnd7\") pod \"maas-controller-6f7b86fcb4-z7zsn\" (UID: \"011f2896-4654-4374-8bab-cc76ec767193\") " pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.603922 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.603765    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/011f2896-4654-4374-8bab-cc76ec767193-webhook-cert\") pod \"maas-controller-6f7b86fcb4-z7zsn\" (UID: \"011f2896-4654-4374-8bab-cc76ec767193\") " pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.606209 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.606178    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/011f2896-4654-4374-8bab-cc76ec767193-webhook-cert\") pod \"maas-controller-6f7b86fcb4-z7zsn\" (UID: \"011f2896-4654-4374-8bab-cc76ec767193\") " pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.612341 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.612319    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxnd7\" (UniqueName: \"kubernetes.io/projected/011f2896-4654-4374-8bab-cc76ec767193-kube-api-access-kxnd7\") pod \"maas-controller-6f7b86fcb4-z7zsn\" (UID: \"011f2896-4654-4374-8bab-cc76ec767193\") " pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.681108 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.681008    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:03.835656 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:23:03.835625    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011f2896_4654_4374_8bab_cc76ec767193.slice/crio-4f7de8c822663b3bd50e76dc239b8c788cdd8100063c67dba30da649821698fb WatchSource:0}: Error finding container 4f7de8c822663b3bd50e76dc239b8c788cdd8100063c67dba30da649821698fb: Status 404 returned error can't find the container with id 4f7de8c822663b3bd50e76dc239b8c788cdd8100063c67dba30da649821698fb
Jul 15 12:23:03.835785 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:03.835695    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f7b86fcb4-z7zsn"]
Jul 15 12:23:04.068374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:04.068335    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67fa5d37-3dd1-44ca-9501-4ea6fb785933" path="/var/lib/kubelet/pods/67fa5d37-3dd1-44ca-9501-4ea6fb785933/volumes"
Jul 15 12:23:04.568841 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:04.568802    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn" event={"ID":"011f2896-4654-4374-8bab-cc76ec767193","Type":"ContainerStarted","Data":"4f7de8c822663b3bd50e76dc239b8c788cdd8100063c67dba30da649821698fb"}
Jul 15 12:23:05.576045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:05.576002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn" event={"ID":"011f2896-4654-4374-8bab-cc76ec767193","Type":"ContainerStarted","Data":"1fea26282d09e9116bcaecb4d11197a7c084b276f95abcd0b2b54939f523d7cf"}
Jul 15 12:23:05.576542 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:05.576063    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:23:05.598360 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:05.598290    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn" podStartSLOduration=1.812651109 podStartE2EDuration="2.59827466s" totalImagesPullingTime="785.623551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:23:05.595606842 +0000 UTC m=+916.695556178" watchObservedRunningTime="2026-07-15 12:23:05.59827466 +0000 UTC m=+916.698223998"
Jul 15 12:23:16.583904 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:23:16.583867    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f7b86fcb4-z7zsn"
Jul 15 12:27:49.451680 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:49.451646    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:27:49.455875 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:49.455848    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:27:52.692023 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:52.691984    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"]
Jul 15 12:27:52.692482 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:52.692201    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn" podUID="47003c97-c574-418d-9cd3-c892956c8a3d" containerName="manager" containerID="cri-o://4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9" gracePeriod=10
Jul 15 12:27:56.562965 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:56.562940    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:27:56.727651 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:56.727615    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mrv9\" (UniqueName: \"kubernetes.io/projected/47003c97-c574-418d-9cd3-c892956c8a3d-kube-api-access-2mrv9\") pod \"47003c97-c574-418d-9cd3-c892956c8a3d\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") "
Jul 15 12:27:56.727822 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:56.727658    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47003c97-c574-418d-9cd3-c892956c8a3d-extensions-socket-volume\") pod \"47003c97-c574-418d-9cd3-c892956c8a3d\" (UID: \"47003c97-c574-418d-9cd3-c892956c8a3d\") "
Jul 15 12:27:56.728059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:56.728032    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47003c97-c574-418d-9cd3-c892956c8a3d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "47003c97-c574-418d-9cd3-c892956c8a3d" (UID: "47003c97-c574-418d-9cd3-c892956c8a3d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:27:56.729746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:56.729725    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47003c97-c574-418d-9cd3-c892956c8a3d-kube-api-access-2mrv9" (OuterVolumeSpecName: "kube-api-access-2mrv9") pod "47003c97-c574-418d-9cd3-c892956c8a3d" (UID: "47003c97-c574-418d-9cd3-c892956c8a3d"). InnerVolumeSpecName "kube-api-access-2mrv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:27:56.828599 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:56.828547    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mrv9\" (UniqueName: \"kubernetes.io/projected/47003c97-c574-418d-9cd3-c892956c8a3d-kube-api-access-2mrv9\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:27:56.828599 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:56.828594    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47003c97-c574-418d-9cd3-c892956c8a3d-extensions-socket-volume\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:27:57.225928 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.225896    2582 generic.go:356] "Generic (PLEG): container finished" podID="47003c97-c574-418d-9cd3-c892956c8a3d" containerID="4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9" exitCode=0
Jul 15 12:27:57.226124 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.225941    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"
Jul 15 12:27:57.226124 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.225966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn" event={"ID":"47003c97-c574-418d-9cd3-c892956c8a3d","Type":"ContainerDied","Data":"4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9"}
Jul 15 12:27:57.226124 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.226007    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn" event={"ID":"47003c97-c574-418d-9cd3-c892956c8a3d","Type":"ContainerDied","Data":"a82e938c86b5f1f934faefe0de9341f09118fb0a21e2db64003838a8ba45df8f"}
Jul 15 12:27:57.226124 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.226027    2582 scope.go:117] "RemoveContainer" containerID="4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9"
Jul 15 12:27:57.236662 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.236643    2582 scope.go:117] "RemoveContainer" containerID="4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9"
Jul 15 12:27:57.236955 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:27:57.236936    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9\": container with ID starting with 4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9 not found: ID does not exist" containerID="4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9"
Jul 15 12:27:57.237008 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.236966    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9"} err="failed to get container status \"4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9\": rpc error: code = NotFound desc = could not find container \"4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9\": container with ID starting with 4fc7fff752a4165f58a6d0271e2b208139e03a5ebbc8f0e6c8d818b9515149f9 not found: ID does not exist"
Jul 15 12:27:57.249082 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.249039    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"]
Jul 15 12:27:57.252741 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:57.252710    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5k6vn"]
Jul 15 12:27:58.062888 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:27:58.062851    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47003c97-c574-418d-9cd3-c892956c8a3d" path="/var/lib/kubelet/pods/47003c97-c574-418d-9cd3-c892956c8a3d/volumes"
Jul 15 12:28:53.838512 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.838479    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"]
Jul 15 12:28:53.838904 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.838874    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47003c97-c574-418d-9cd3-c892956c8a3d" containerName="manager"
Jul 15 12:28:53.838904 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.838885    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="47003c97-c574-418d-9cd3-c892956c8a3d" containerName="manager"
Jul 15 12:28:53.838983 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.838951    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="47003c97-c574-418d-9cd3-c892956c8a3d" containerName="manager"
Jul 15 12:28:53.843162 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.843144    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:53.845677 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.845655    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vjlzw\""
Jul 15 12:28:53.853720 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.853695    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"]
Jul 15 12:28:53.918983 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.918934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rjj8\" (UniqueName: \"kubernetes.io/projected/695dde12-6708-44bd-9d58-80d6d0b671ec-kube-api-access-4rjj8\") pod \"kuadrant-operator-controller-manager-86878c769f-ttdlv\" (UID: \"695dde12-6708-44bd-9d58-80d6d0b671ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:53.919208 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:53.919027    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/695dde12-6708-44bd-9d58-80d6d0b671ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ttdlv\" (UID: \"695dde12-6708-44bd-9d58-80d6d0b671ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:54.020242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.020202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rjj8\" (UniqueName: \"kubernetes.io/projected/695dde12-6708-44bd-9d58-80d6d0b671ec-kube-api-access-4rjj8\") pod \"kuadrant-operator-controller-manager-86878c769f-ttdlv\" (UID: \"695dde12-6708-44bd-9d58-80d6d0b671ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:54.020416 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.020279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/695dde12-6708-44bd-9d58-80d6d0b671ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ttdlv\" (UID: \"695dde12-6708-44bd-9d58-80d6d0b671ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:54.020631 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.020615    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/695dde12-6708-44bd-9d58-80d6d0b671ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ttdlv\" (UID: \"695dde12-6708-44bd-9d58-80d6d0b671ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:54.034834 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.034804    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rjj8\" (UniqueName: \"kubernetes.io/projected/695dde12-6708-44bd-9d58-80d6d0b671ec-kube-api-access-4rjj8\") pod \"kuadrant-operator-controller-manager-86878c769f-ttdlv\" (UID: \"695dde12-6708-44bd-9d58-80d6d0b671ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:54.154187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.154072    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:54.300686 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.300663    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"]
Jul 15 12:28:54.303109 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:28:54.303065    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod695dde12_6708_44bd_9d58_80d6d0b671ec.slice/crio-ab8ac9a7afea7cd9c473e662c72515168b9749437223160165829c63a581eac2 WatchSource:0}: Error finding container ab8ac9a7afea7cd9c473e662c72515168b9749437223160165829c63a581eac2: Status 404 returned error can't find the container with id ab8ac9a7afea7cd9c473e662c72515168b9749437223160165829c63a581eac2
Jul 15 12:28:54.536088 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.536050    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv" event={"ID":"695dde12-6708-44bd-9d58-80d6d0b671ec","Type":"ContainerStarted","Data":"75001248af84c8555ac4f471f5813b50a0b7013b38347b5a51b0cb265e2f2fd7"}
Jul 15 12:28:54.536295 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.536234    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:28:54.536295 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.536266    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv" event={"ID":"695dde12-6708-44bd-9d58-80d6d0b671ec","Type":"ContainerStarted","Data":"ab8ac9a7afea7cd9c473e662c72515168b9749437223160165829c63a581eac2"}
Jul 15 12:28:54.569605 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:28:54.569553    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv" podStartSLOduration=1.5695365159999999 podStartE2EDuration="1.569536516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:54.567694672 +0000 UTC m=+1265.667644012" watchObservedRunningTime="2026-07-15 12:28:54.569536516 +0000 UTC m=+1265.669485854"
Jul 15 12:29:05.543587 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:29:05.543555    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ttdlv"
Jul 15 12:30:00.144354 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.144318    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf"]
Jul 15 12:30:00.147433 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.147417    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf"
Jul 15 12:30:00.149391 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.149369    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:30:00.154061 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.154034    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf"]
Jul 15 12:30:00.159055 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.159036    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf"
Jul 15 12:30:00.303610 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.303575    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf"]
Jul 15 12:30:00.305834 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:30:00.305811    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21b82ed_0447_485d_a6e1_522b0faa4b99.slice/crio-00cb559c165a1ec0a78cd5a9e0b8e4a425256dd74538536603d52d29a2ca2974 WatchSource:0}: Error finding container 00cb559c165a1ec0a78cd5a9e0b8e4a425256dd74538536603d52d29a2ca2974: Status 404 returned error can't find the container with id 00cb559c165a1ec0a78cd5a9e0b8e4a425256dd74538536603d52d29a2ca2974
Jul 15 12:30:00.308126 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.308086    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:30:00.903901 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:00.903867    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf" event={"ID":"b21b82ed-0447-485d-a6e1-522b0faa4b99","Type":"ContainerStarted","Data":"00cb559c165a1ec0a78cd5a9e0b8e4a425256dd74538536603d52d29a2ca2974"}
Jul 15 12:30:05.936033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:05.935989    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf" event={"ID":"b21b82ed-0447-485d-a6e1-522b0faa4b99","Type":"ContainerStarted","Data":"d4f689cdcba151dc02a769abe8e67dd520f5db1be8ce11d02494e2c1d3ac0b4d"}
Jul 15 12:30:05.958722 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:05.958654    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf" podStartSLOduration=0.793538914 podStartE2EDuration="5.958636511s" totalImagesPullingTime="5.165097597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:05.956540532 +0000 UTC m=+1337.056489872" watchObservedRunningTime="2026-07-15 12:30:05.958636511 +0000 UTC m=+1337.058585851"
Jul 15 12:30:06.942862 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:06.942772    2582 generic.go:356] "Generic (PLEG): container finished" podID="b21b82ed-0447-485d-a6e1-522b0faa4b99" containerID="d4f689cdcba151dc02a769abe8e67dd520f5db1be8ce11d02494e2c1d3ac0b4d" exitCode=0
Jul 15 12:30:06.943266 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:06.942853    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf" event={"ID":"b21b82ed-0447-485d-a6e1-522b0faa4b99","Type":"ContainerDied","Data":"d4f689cdcba151dc02a769abe8e67dd520f5db1be8ce11d02494e2c1d3ac0b4d"}
Jul 15 12:30:08.089444 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:08.089420    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf"
Jul 15 12:30:08.954488 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:08.954460    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf"
Jul 15 12:30:08.954661 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:08.954496    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735310-nxzcf" event={"ID":"b21b82ed-0447-485d-a6e1-522b0faa4b99","Type":"ContainerDied","Data":"00cb559c165a1ec0a78cd5a9e0b8e4a425256dd74538536603d52d29a2ca2974"}
Jul 15 12:30:08.954661 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:30:08.954545    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00cb559c165a1ec0a78cd5a9e0b8e4a425256dd74538536603d52d29a2ca2974"
Jul 15 12:32:49.487766 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:32:49.487736    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:32:49.498475 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:32:49.498450    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:34:57.878585 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.878550    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"]
Jul 15 12:34:57.879170 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.879006    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b21b82ed-0447-485d-a6e1-522b0faa4b99" containerName="cleanup"
Jul 15 12:34:57.879170 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.879021    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b21b82ed-0447-485d-a6e1-522b0faa4b99" containerName="cleanup"
Jul 15 12:34:57.879170 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.879115    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b21b82ed-0447-485d-a6e1-522b0faa4b99" containerName="cleanup"
Jul 15 12:34:57.882324 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.882304    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:57.884812 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.884790    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a506c9f6-serving-cert\""
Jul 15 12:34:57.885237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.885219    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:34:57.892822 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:57.892795    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"]
Jul 15 12:34:58.021390 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.021359    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-maas-api-tls\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.021557 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.021431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkjsb\" (UniqueName: \"kubernetes.io/projected/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-kube-api-access-nkjsb\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.021557 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.021459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-openshift-service-ca\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.122337 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.122303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkjsb\" (UniqueName: \"kubernetes.io/projected/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-kube-api-access-nkjsb\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.122504 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.122348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-openshift-service-ca\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.122548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.122515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-maas-api-tls\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.122960 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.122941    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-openshift-service-ca\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.124865 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.124844    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-maas-api-tls\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.130102 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.130035    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkjsb\" (UniqueName: \"kubernetes.io/projected/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-kube-api-access-nkjsb\") pod \"maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.193144 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.193120    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:58.332064 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.331990    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"]
Jul 15 12:34:58.333992 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:34:58.333949    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e46a7dd_827b_4799_85dd_405a6ed1cf4e.slice/crio-c30f02920dee16e6e3aafa8d867e7ad71ce3eb5a4cf6bfd3eaacea07aa6b5e0a WatchSource:0}: Error finding container c30f02920dee16e6e3aafa8d867e7ad71ce3eb5a4cf6bfd3eaacea07aa6b5e0a: Status 404 returned error can't find the container with id c30f02920dee16e6e3aafa8d867e7ad71ce3eb5a4cf6bfd3eaacea07aa6b5e0a
Jul 15 12:34:58.622722 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:58.622684    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d" event={"ID":"1e46a7dd-827b-4799-85dd-405a6ed1cf4e","Type":"ContainerStarted","Data":"c30f02920dee16e6e3aafa8d867e7ad71ce3eb5a4cf6bfd3eaacea07aa6b5e0a"}
Jul 15 12:34:59.629956 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:59.629871    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d" event={"ID":"1e46a7dd-827b-4799-85dd-405a6ed1cf4e","Type":"ContainerStarted","Data":"7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821"}
Jul 15 12:34:59.630340 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:59.629976    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:34:59.649192 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:34:59.649142    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d" podStartSLOduration=1.666770192 podStartE2EDuration="2.64912733s" totalImagesPullingTime="982.357138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:34:59.646176994 +0000 UTC m=+1630.746126361" watchObservedRunningTime="2026-07-15 12:34:59.64912733 +0000 UTC m=+1630.749076667"
Jul 15 12:35:02.968301 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:02.968263    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"]
Jul 15 12:35:02.971948 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:02.971931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"
Jul 15 12:35:02.974150 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:02.974129    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:35:02.980830 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:02.979814    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"]
Jul 15 12:35:03.066892 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:03.066861    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d\" (UID: \"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"
Jul 15 12:35:03.167676 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:03.167640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d\" (UID: \"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"
Jul 15 12:35:03.168272 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:03.168255    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d\" (UID: \"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"
Jul 15 12:35:03.285072 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:03.284986    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"
Jul 15 12:35:03.425765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:03.425741    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"]
Jul 15 12:35:03.427541 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:35:03.427516    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c080f8c_ccdf_4710_b8f4_9aa0f13a2b66.slice/crio-7254a6e520b2fd4716b52390180b98cd94d433d51701898fc1ca6fef531830c4 WatchSource:0}: Error finding container 7254a6e520b2fd4716b52390180b98cd94d433d51701898fc1ca6fef531830c4: Status 404 returned error can't find the container with id 7254a6e520b2fd4716b52390180b98cd94d433d51701898fc1ca6fef531830c4
Jul 15 12:35:03.660088 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:03.660054    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d" event={"ID":"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66","Type":"ContainerStarted","Data":"7254a6e520b2fd4716b52390180b98cd94d433d51701898fc1ca6fef531830c4"}
Jul 15 12:35:04.667801 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:04.667763    2582 generic.go:356] "Generic (PLEG): container finished" podID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerID="c2620f924ca1c8cded29730e55b3b3c2566852c5dbed59a35282774cb2343b3d" exitCode=7
Jul 15 12:35:04.668215 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:04.667849    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d" event={"ID":"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66","Type":"ContainerDied","Data":"c2620f924ca1c8cded29730e55b3b3c2566852c5dbed59a35282774cb2343b3d"}
Jul 15 12:35:04.668215 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:04.668170    2582 scope.go:117] "RemoveContainer" containerID="c2620f924ca1c8cded29730e55b3b3c2566852c5dbed59a35282774cb2343b3d"
Jul 15 12:35:05.641520 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:05.641437    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:35:05.675455 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:05.675406    2582 generic.go:356] "Generic (PLEG): container finished" podID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerID="4a19f3286c039541b11ce511a74f01318a4722a722a0017ff2fa807ce757f0ce" exitCode=7
Jul 15 12:35:05.675936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:05.675488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d" event={"ID":"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66","Type":"ContainerDied","Data":"4a19f3286c039541b11ce511a74f01318a4722a722a0017ff2fa807ce757f0ce"}
Jul 15 12:35:05.675936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:05.675557    2582 scope.go:117] "RemoveContainer" containerID="c2620f924ca1c8cded29730e55b3b3c2566852c5dbed59a35282774cb2343b3d"
Jul 15 12:35:05.675936 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:05.675791    2582 scope.go:117] "RemoveContainer" containerID="4a19f3286c039541b11ce511a74f01318a4722a722a0017ff2fa807ce757f0ce"
Jul 15 12:35:05.676072 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:35:05.676048    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d_odh-ai-gateway-infra(7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66"
Jul 15 12:35:06.682708 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:06.682684    2582 scope.go:117] "RemoveContainer" containerID="4a19f3286c039541b11ce511a74f01318a4722a722a0017ff2fa807ce757f0ce"
Jul 15 12:35:06.683083 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:35:06.682887    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d_odh-ai-gateway-infra(7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66"
Jul 15 12:35:20.060080 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:20.060050    2582 scope.go:117] "RemoveContainer" containerID="4a19f3286c039541b11ce511a74f01318a4722a722a0017ff2fa807ce757f0ce"
Jul 15 12:35:20.769042 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:20.769009    2582 generic.go:356] "Generic (PLEG): container finished" podID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerID="02978ccc6cc0c0b717045d219e39596554f8ea650f548772cd5c2c567f2724fb" exitCode=0
Jul 15 12:35:20.769198 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:20.769078    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d" event={"ID":"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66","Type":"ContainerDied","Data":"02978ccc6cc0c0b717045d219e39596554f8ea650f548772cd5c2c567f2724fb"}
Jul 15 12:35:20.769198 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:20.769158    2582 scope.go:117] "RemoveContainer" containerID="4a19f3286c039541b11ce511a74f01318a4722a722a0017ff2fa807ce757f0ce"
Jul 15 12:35:21.807151 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:21.807117    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"]
Jul 15 12:35:21.919924 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:21.919903    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"
Jul 15 12:35:21.928395 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:21.928372    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66-maas-api-service-ca\") pod \"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66\" (UID: \"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66\") "
Jul 15 12:35:21.928709 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:21.928685    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" (UID: "7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:35:22.029033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:22.028957    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:35:22.783775 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:22.783745    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"
Jul 15 12:35:22.783948 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:22.783786    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d" event={"ID":"7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66","Type":"ContainerDied","Data":"7254a6e520b2fd4716b52390180b98cd94d433d51701898fc1ca6fef531830c4"}
Jul 15 12:35:22.783948 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:22.783837    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7254a6e520b2fd4716b52390180b98cd94d433d51701898fc1ca6fef531830c4"
Jul 15 12:35:22.819475 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:22.819445    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"]
Jul 15 12:35:22.829386 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:22.829360    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-w7s5d"]
Jul 15 12:35:24.063373 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:24.063337    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" path="/var/lib/kubelet/pods/7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66/volumes"
Jul 15 12:35:53.019987 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.019949    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"]
Jul 15 12:35:53.020478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020362    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020374    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020385    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020391    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020402    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020408    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020478    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020699 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020486    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.020699 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.020491    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c080f8c-ccdf-4710-b8f4-9aa0f13a2b66" containerName="revoke-keys"
Jul 15 12:35:53.023408 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.023391    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"
Jul 15 12:35:53.025780 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.025760    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:35:53.031839 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.031806    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"]
Jul 15 12:35:53.112730 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.112690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62a707e-f568-43e6-8add-5e8867a0a8d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f\" (UID: \"f62a707e-f568-43e6-8add-5e8867a0a8d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"
Jul 15 12:35:53.213799 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.213760    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62a707e-f568-43e6-8add-5e8867a0a8d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f\" (UID: \"f62a707e-f568-43e6-8add-5e8867a0a8d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"
Jul 15 12:35:53.214501 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.214481    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62a707e-f568-43e6-8add-5e8867a0a8d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f\" (UID: \"f62a707e-f568-43e6-8add-5e8867a0a8d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"
Jul 15 12:35:53.335245 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.335147    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"
Jul 15 12:35:53.489830 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.489801    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"]
Jul 15 12:35:53.492376 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:35:53.492352    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62a707e_f568_43e6_8add_5e8867a0a8d8.slice/crio-fb9c151d6b19f66056f9494d73e0f695728197642c29e9938f12f80053d03c60 WatchSource:0}: Error finding container fb9c151d6b19f66056f9494d73e0f695728197642c29e9938f12f80053d03c60: Status 404 returned error can't find the container with id fb9c151d6b19f66056f9494d73e0f695728197642c29e9938f12f80053d03c60
Jul 15 12:35:53.958618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.958575    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f" event={"ID":"f62a707e-f568-43e6-8add-5e8867a0a8d8","Type":"ContainerStarted","Data":"b08cbfb467f5354647628dfc9e733c4b14aee9fcacd498f6624c2a9d5e6fd4c1"}
Jul 15 12:35:53.958816 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.958632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f" event={"ID":"f62a707e-f568-43e6-8add-5e8867a0a8d8","Type":"ContainerStarted","Data":"fb9c151d6b19f66056f9494d73e0f695728197642c29e9938f12f80053d03c60"}
Jul 15 12:35:53.977938 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:53.977881    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f" podStartSLOduration=0.977856226 podStartE2EDuration="977.856226ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:35:53.977611406 +0000 UTC m=+1685.077560757" watchObservedRunningTime="2026-07-15 12:35:53.977856226 +0000 UTC m=+1685.077805564"
Jul 15 12:35:54.965633 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:54.965595    2582 generic.go:356] "Generic (PLEG): container finished" podID="f62a707e-f568-43e6-8add-5e8867a0a8d8" containerID="b08cbfb467f5354647628dfc9e733c4b14aee9fcacd498f6624c2a9d5e6fd4c1" exitCode=0
Jul 15 12:35:54.966023 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:54.965685    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f" event={"ID":"f62a707e-f568-43e6-8add-5e8867a0a8d8","Type":"ContainerDied","Data":"b08cbfb467f5354647628dfc9e733c4b14aee9fcacd498f6624c2a9d5e6fd4c1"}
Jul 15 12:35:56.103696 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:56.103673    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"
Jul 15 12:35:56.241244 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:56.241163    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62a707e-f568-43e6-8add-5e8867a0a8d8-maas-api-service-ca\") pod \"f62a707e-f568-43e6-8add-5e8867a0a8d8\" (UID: \"f62a707e-f568-43e6-8add-5e8867a0a8d8\") "
Jul 15 12:35:56.241511 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:56.241489    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f62a707e-f568-43e6-8add-5e8867a0a8d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f62a707e-f568-43e6-8add-5e8867a0a8d8" (UID: "f62a707e-f568-43e6-8add-5e8867a0a8d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:35:56.342530 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:56.342493    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f62a707e-f568-43e6-8add-5e8867a0a8d8-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:35:56.977667 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:56.977635    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f"
Jul 15 12:35:56.977845 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:56.977666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f" event={"ID":"f62a707e-f568-43e6-8add-5e8867a0a8d8","Type":"ContainerDied","Data":"fb9c151d6b19f66056f9494d73e0f695728197642c29e9938f12f80053d03c60"}
Jul 15 12:35:56.977845 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:35:56.977722    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb9c151d6b19f66056f9494d73e0f695728197642c29e9938f12f80053d03c60"
Jul 15 12:36:03.578781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:03.578744    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"]
Jul 15 12:36:03.579272 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:03.579017    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d" podUID="1e46a7dd-827b-4799-85dd-405a6ed1cf4e" containerName="maas-api" containerID="cri-o://7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821" gracePeriod=30
Jul 15 12:36:03.831382 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:03.831309    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:36:04.011777 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.011739    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkjsb\" (UniqueName: \"kubernetes.io/projected/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-kube-api-access-nkjsb\") pod \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") "
Jul 15 12:36:04.011962 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.011787    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-maas-api-tls\") pod \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") "
Jul 15 12:36:04.011962 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.011875    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-openshift-service-ca\") pod \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\" (UID: \"1e46a7dd-827b-4799-85dd-405a6ed1cf4e\") "
Jul 15 12:36:04.012324 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.012293    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e46a7dd-827b-4799-85dd-405a6ed1cf4e" (UID: "1e46a7dd-827b-4799-85dd-405a6ed1cf4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:36:04.014059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.014032    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-kube-api-access-nkjsb" (OuterVolumeSpecName: "kube-api-access-nkjsb") pod "1e46a7dd-827b-4799-85dd-405a6ed1cf4e" (UID: "1e46a7dd-827b-4799-85dd-405a6ed1cf4e"). InnerVolumeSpecName "kube-api-access-nkjsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:36:04.014059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.014044    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e46a7dd-827b-4799-85dd-405a6ed1cf4e" (UID: "1e46a7dd-827b-4799-85dd-405a6ed1cf4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:36:04.021989 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.021958    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e46a7dd-827b-4799-85dd-405a6ed1cf4e" containerID="7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821" exitCode=0
Jul 15 12:36:04.022079 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.021997    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"
Jul 15 12:36:04.022079 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.022037    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d" event={"ID":"1e46a7dd-827b-4799-85dd-405a6ed1cf4e","Type":"ContainerDied","Data":"7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821"}
Jul 15 12:36:04.022177 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.022088    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d" event={"ID":"1e46a7dd-827b-4799-85dd-405a6ed1cf4e","Type":"ContainerDied","Data":"c30f02920dee16e6e3aafa8d867e7ad71ce3eb5a4cf6bfd3eaacea07aa6b5e0a"}
Jul 15 12:36:04.022177 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.022131    2582 scope.go:117] "RemoveContainer" containerID="7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821"
Jul 15 12:36:04.035026 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.035001    2582 scope.go:117] "RemoveContainer" containerID="7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821"
Jul 15 12:36:04.035347 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:36:04.035327    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821\": container with ID starting with 7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821 not found: ID does not exist" containerID="7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821"
Jul 15 12:36:04.035431 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.035360    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821"} err="failed to get container status \"7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821\": rpc error: code = NotFound desc = could not find container \"7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821\": container with ID starting with 7a9d3ed0fce7ee1172c2d37f9ed684c07b452c23dbc2a5a5f1a97e67c6401821 not found: ID does not exist"
Jul 15 12:36:04.046903 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.046873    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"]
Jul 15 12:36:04.051944 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.051909    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a506c9f6-7d58d9d7f4-gxd7d"]
Jul 15 12:36:04.062799 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.062772    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e46a7dd-827b-4799-85dd-405a6ed1cf4e" path="/var/lib/kubelet/pods/1e46a7dd-827b-4799-85dd-405a6ed1cf4e/volumes"
Jul 15 12:36:04.112729 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.112635    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:36:04.112729 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.112669    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkjsb\" (UniqueName: \"kubernetes.io/projected/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-kube-api-access-nkjsb\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:36:04.112729 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:04.112681    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e46a7dd-827b-4799-85dd-405a6ed1cf4e-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:36:20.322918 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.322882    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"]
Jul 15 12:36:20.323500 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.323479    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f62a707e-f568-43e6-8add-5e8867a0a8d8" containerName="revoke-keys"
Jul 15 12:36:20.323500 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.323500    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f62a707e-f568-43e6-8add-5e8867a0a8d8" containerName="revoke-keys"
Jul 15 12:36:20.323646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.323539    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e46a7dd-827b-4799-85dd-405a6ed1cf4e" containerName="maas-api"
Jul 15 12:36:20.323646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.323548    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e46a7dd-827b-4799-85dd-405a6ed1cf4e" containerName="maas-api"
Jul 15 12:36:20.323646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.323632    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f62a707e-f568-43e6-8add-5e8867a0a8d8" containerName="revoke-keys"
Jul 15 12:36:20.323646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.323646    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e46a7dd-827b-4799-85dd-405a6ed1cf4e" containerName="maas-api"
Jul 15 12:36:20.328390 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.328371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.330860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.330838    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-ba8973be-serving-cert\""
Jul 15 12:36:20.332141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.332118    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:36:20.335312 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.335292    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"]
Jul 15 12:36:20.348440 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.348409    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-openshift-service-ca\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.348546 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.348473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-maas-api-tls\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.348546 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.348524    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl2f6\" (UniqueName: \"kubernetes.io/projected/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-kube-api-access-sl2f6\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.449524 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.449489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-openshift-service-ca\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.449664 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.449534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-maas-api-tls\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.449664 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.449565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2f6\" (UniqueName: \"kubernetes.io/projected/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-kube-api-access-sl2f6\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.450203 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.450183    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-openshift-service-ca\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.452006 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.451981    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-maas-api-tls\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.457161 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.457139    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl2f6\" (UniqueName: \"kubernetes.io/projected/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-kube-api-access-sl2f6\") pod \"maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.640359 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.640268    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:20.787296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.787258    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"]
Jul 15 12:36:20.788845 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:36:20.788820    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd048be5d_4776_4e0a_a9b0_65bb2f7fe3d6.slice/crio-c3371b18a25a9aae6433ee0b6580ee59724d5e00c0c810dd5917190081b56c38 WatchSource:0}: Error finding container c3371b18a25a9aae6433ee0b6580ee59724d5e00c0c810dd5917190081b56c38: Status 404 returned error can't find the container with id c3371b18a25a9aae6433ee0b6580ee59724d5e00c0c810dd5917190081b56c38
Jul 15 12:36:20.790951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:20.790930    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:36:21.120031 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:21.119993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk" event={"ID":"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6","Type":"ContainerStarted","Data":"c3371b18a25a9aae6433ee0b6580ee59724d5e00c0c810dd5917190081b56c38"}
Jul 15 12:36:24.140182 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.140138    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk" event={"ID":"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6","Type":"ContainerStarted","Data":"632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65"}
Jul 15 12:36:24.140773 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.140269    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:24.161172 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.161113    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk" podStartSLOduration=1.207154934 podStartE2EDuration="4.161085463s" totalImagesPullingTime="2.953930529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:36:24.158969521 +0000 UTC m=+1715.258918861" watchObservedRunningTime="2026-07-15 12:36:24.161085463 +0000 UTC m=+1715.261034800"
Jul 15 12:36:24.714400 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.714370    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"]
Jul 15 12:36:24.717984 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.717967    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"
Jul 15 12:36:24.720570 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.720546    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:36:24.727649 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.727628    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"]
Jul 15 12:36:24.795240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.795204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbabcaba-6143-47d8-a63f-d0f5b7362bcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8\" (UID: \"cbabcaba-6143-47d8-a63f-d0f5b7362bcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"
Jul 15 12:36:24.895748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.895705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbabcaba-6143-47d8-a63f-d0f5b7362bcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8\" (UID: \"cbabcaba-6143-47d8-a63f-d0f5b7362bcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"
Jul 15 12:36:24.896342 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:24.896324    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbabcaba-6143-47d8-a63f-d0f5b7362bcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8\" (UID: \"cbabcaba-6143-47d8-a63f-d0f5b7362bcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"
Jul 15 12:36:25.028836 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:25.028753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"
Jul 15 12:36:25.186749 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:25.186595    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"]
Jul 15 12:36:25.189516 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:36:25.189494    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbabcaba_6143_47d8_a63f_d0f5b7362bcb.slice/crio-b0cec6258681858140caddf057fe08648dc738b72b0fe7a6799380993e5ec949 WatchSource:0}: Error finding container b0cec6258681858140caddf057fe08648dc738b72b0fe7a6799380993e5ec949: Status 404 returned error can't find the container with id b0cec6258681858140caddf057fe08648dc738b72b0fe7a6799380993e5ec949
Jul 15 12:36:26.156893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:26.156793    2582 generic.go:356] "Generic (PLEG): container finished" podID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerID="6f39cd5e2e0a20b97b3448e94744b937b9896c94de0c7335283804d3534f1553" exitCode=7
Jul 15 12:36:26.156893 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:26.156834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8" event={"ID":"cbabcaba-6143-47d8-a63f-d0f5b7362bcb","Type":"ContainerDied","Data":"6f39cd5e2e0a20b97b3448e94744b937b9896c94de0c7335283804d3534f1553"}
Jul 15 12:36:26.157133 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:26.156902    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8" event={"ID":"cbabcaba-6143-47d8-a63f-d0f5b7362bcb","Type":"ContainerStarted","Data":"b0cec6258681858140caddf057fe08648dc738b72b0fe7a6799380993e5ec949"}
Jul 15 12:36:26.157193 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:26.157163    2582 scope.go:117] "RemoveContainer" containerID="6f39cd5e2e0a20b97b3448e94744b937b9896c94de0c7335283804d3534f1553"
Jul 15 12:36:27.163956 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:27.163864    2582 generic.go:356] "Generic (PLEG): container finished" podID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerID="89d5907b4ffcb811ba79ee11698e39115c7082b97da122d920211719c280828f" exitCode=7
Jul 15 12:36:27.164414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:27.163953    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8" event={"ID":"cbabcaba-6143-47d8-a63f-d0f5b7362bcb","Type":"ContainerDied","Data":"89d5907b4ffcb811ba79ee11698e39115c7082b97da122d920211719c280828f"}
Jul 15 12:36:27.164414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:27.164023    2582 scope.go:117] "RemoveContainer" containerID="6f39cd5e2e0a20b97b3448e94744b937b9896c94de0c7335283804d3534f1553"
Jul 15 12:36:27.164414 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:27.164264    2582 scope.go:117] "RemoveContainer" containerID="89d5907b4ffcb811ba79ee11698e39115c7082b97da122d920211719c280828f"
Jul 15 12:36:27.164545 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:36:27.164475    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8_odh-ai-gateway-infra(cbabcaba-6143-47d8-a63f-d0f5b7362bcb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb"
Jul 15 12:36:28.171596 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:28.171567    2582 scope.go:117] "RemoveContainer" containerID="89d5907b4ffcb811ba79ee11698e39115c7082b97da122d920211719c280828f"
Jul 15 12:36:28.171974 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:36:28.171760    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8_odh-ai-gateway-infra(cbabcaba-6143-47d8-a63f-d0f5b7362bcb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb"
Jul 15 12:36:30.155331 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:30.155300    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:36:41.056790 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:41.056757    2582 scope.go:117] "RemoveContainer" containerID="89d5907b4ffcb811ba79ee11698e39115c7082b97da122d920211719c280828f"
Jul 15 12:36:42.258045 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:42.258009    2582 generic.go:356] "Generic (PLEG): container finished" podID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerID="d8f5ded5ea89d51b6b95a7ac67df8b0dc4353bc04235e765312efef417267e82" exitCode=0
Jul 15 12:36:42.258440 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:42.258122    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8" event={"ID":"cbabcaba-6143-47d8-a63f-d0f5b7362bcb","Type":"ContainerDied","Data":"d8f5ded5ea89d51b6b95a7ac67df8b0dc4353bc04235e765312efef417267e82"}
Jul 15 12:36:42.258440 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:42.258220    2582 scope.go:117] "RemoveContainer" containerID="89d5907b4ffcb811ba79ee11698e39115c7082b97da122d920211719c280828f"
Jul 15 12:36:43.290562 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:43.290529    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"]
Jul 15 12:36:43.403406 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:43.403382    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"
Jul 15 12:36:43.561247 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:43.561161    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbabcaba-6143-47d8-a63f-d0f5b7362bcb-maas-api-service-ca\") pod \"cbabcaba-6143-47d8-a63f-d0f5b7362bcb\" (UID: \"cbabcaba-6143-47d8-a63f-d0f5b7362bcb\") "
Jul 15 12:36:43.561526 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:43.561507    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbabcaba-6143-47d8-a63f-d0f5b7362bcb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cbabcaba-6143-47d8-a63f-d0f5b7362bcb" (UID: "cbabcaba-6143-47d8-a63f-d0f5b7362bcb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:36:43.662586 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:43.662547    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbabcaba-6143-47d8-a63f-d0f5b7362bcb-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:36:44.271631 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:44.271599    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"
Jul 15 12:36:44.271801 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:44.271639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8" event={"ID":"cbabcaba-6143-47d8-a63f-d0f5b7362bcb","Type":"ContainerDied","Data":"b0cec6258681858140caddf057fe08648dc738b72b0fe7a6799380993e5ec949"}
Jul 15 12:36:44.271801 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:44.271684    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0cec6258681858140caddf057fe08648dc738b72b0fe7a6799380993e5ec949"
Jul 15 12:36:44.298306 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:44.298271    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"]
Jul 15 12:36:44.306833 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:44.306805    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-77xr8"]
Jul 15 12:36:46.063354 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:36:46.063320    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" path="/var/lib/kubelet/pods/cbabcaba-6143-47d8-a63f-d0f5b7362bcb/volumes"
Jul 15 12:37:14.758267 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758231    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"]
Jul 15 12:37:14.758746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758635    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.758746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758646    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.758746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758663    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.758746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758669    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.758746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758679    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.758746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758685    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.758746 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758747    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.758984 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.758759    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:14.762104 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.762074    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"
Jul 15 12:37:14.764155 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.764129    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:37:14.769577 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.769549    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"]
Jul 15 12:37:14.822217 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.822186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1419b3a-6a83-4fc7-a7e4-0eae612f206a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk\" (UID: \"a1419b3a-6a83-4fc7-a7e4-0eae612f206a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"
Jul 15 12:37:14.922890 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.922850    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1419b3a-6a83-4fc7-a7e4-0eae612f206a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk\" (UID: \"a1419b3a-6a83-4fc7-a7e4-0eae612f206a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"
Jul 15 12:37:14.923621 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:14.923599    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1419b3a-6a83-4fc7-a7e4-0eae612f206a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk\" (UID: \"a1419b3a-6a83-4fc7-a7e4-0eae612f206a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"
Jul 15 12:37:15.074644 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:15.074558    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"
Jul 15 12:37:15.211725 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:15.211694    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"]
Jul 15 12:37:15.213077 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:37:15.213047    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1419b3a_6a83_4fc7_a7e4_0eae612f206a.slice/crio-8c491260be267c3f5a9a8f2cb9abdb8df551b99d2e751644ec6ab32089f7566f WatchSource:0}: Error finding container 8c491260be267c3f5a9a8f2cb9abdb8df551b99d2e751644ec6ab32089f7566f: Status 404 returned error can't find the container with id 8c491260be267c3f5a9a8f2cb9abdb8df551b99d2e751644ec6ab32089f7566f
Jul 15 12:37:15.446396 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:15.446362    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk" event={"ID":"a1419b3a-6a83-4fc7-a7e4-0eae612f206a","Type":"ContainerStarted","Data":"8c491260be267c3f5a9a8f2cb9abdb8df551b99d2e751644ec6ab32089f7566f"}
Jul 15 12:37:16.453561 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:16.453528    2582 generic.go:356] "Generic (PLEG): container finished" podID="a1419b3a-6a83-4fc7-a7e4-0eae612f206a" containerID="35b2ef6507dba1213a2f1da8b203c344b639623745790c8e0ebc79d6494fdf0e" exitCode=0
Jul 15 12:37:16.453963 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:16.453618    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk" event={"ID":"a1419b3a-6a83-4fc7-a7e4-0eae612f206a","Type":"ContainerDied","Data":"35b2ef6507dba1213a2f1da8b203c344b639623745790c8e0ebc79d6494fdf0e"}
Jul 15 12:37:17.594911 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:17.594889    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"
Jul 15 12:37:17.646280 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:17.646244    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1419b3a-6a83-4fc7-a7e4-0eae612f206a-maas-api-service-ca\") pod \"a1419b3a-6a83-4fc7-a7e4-0eae612f206a\" (UID: \"a1419b3a-6a83-4fc7-a7e4-0eae612f206a\") "
Jul 15 12:37:17.646596 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:17.646574    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1419b3a-6a83-4fc7-a7e4-0eae612f206a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a1419b3a-6a83-4fc7-a7e4-0eae612f206a" (UID: "a1419b3a-6a83-4fc7-a7e4-0eae612f206a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:37:17.747477 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:17.747394    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1419b3a-6a83-4fc7-a7e4-0eae612f206a-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:37:18.466265 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:18.466236    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk"
Jul 15 12:37:18.466443 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:18.466284    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk" event={"ID":"a1419b3a-6a83-4fc7-a7e4-0eae612f206a","Type":"ContainerDied","Data":"8c491260be267c3f5a9a8f2cb9abdb8df551b99d2e751644ec6ab32089f7566f"}
Jul 15 12:37:18.466443 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:18.466337    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c491260be267c3f5a9a8f2cb9abdb8df551b99d2e751644ec6ab32089f7566f"
Jul 15 12:37:25.416201 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.416165    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"]
Jul 15 12:37:25.416672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.416507    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk" podUID="d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" containerName="maas-api" containerID="cri-o://632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65" gracePeriod=30
Jul 15 12:37:25.691934 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.691904    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:37:25.718923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.718896    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl2f6\" (UniqueName: \"kubernetes.io/projected/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-kube-api-access-sl2f6\") pod \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") "
Jul 15 12:37:25.718923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.718927    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-maas-api-tls\") pod \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") "
Jul 15 12:37:25.719140 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.718967    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-openshift-service-ca\") pod \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\" (UID: \"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6\") "
Jul 15 12:37:25.719343 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.719319    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" (UID: "d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:37:25.720953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.720927    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-kube-api-access-sl2f6" (OuterVolumeSpecName: "kube-api-access-sl2f6") pod "d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" (UID: "d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6"). InnerVolumeSpecName "kube-api-access-sl2f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:37:25.721053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.720956    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" (UID: "d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:37:25.820018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.819979    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl2f6\" (UniqueName: \"kubernetes.io/projected/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-kube-api-access-sl2f6\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:37:25.820018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.820012    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:37:25.820018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:25.820021    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:37:26.516424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.516385    2582 generic.go:356] "Generic (PLEG): container finished" podID="d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" containerID="632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65" exitCode=0
Jul 15 12:37:26.517005 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.516978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk" event={"ID":"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6","Type":"ContainerDied","Data":"632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65"}
Jul 15 12:37:26.517219 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.517200    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk" event={"ID":"d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6","Type":"ContainerDied","Data":"c3371b18a25a9aae6433ee0b6580ee59724d5e00c0c810dd5917190081b56c38"}
Jul 15 12:37:26.517359 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.517344    2582 scope.go:117] "RemoveContainer" containerID="632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65"
Jul 15 12:37:26.517649 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.517634    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"
Jul 15 12:37:26.527987 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.527970    2582 scope.go:117] "RemoveContainer" containerID="632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65"
Jul 15 12:37:26.528235 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:37:26.528217    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65\": container with ID starting with 632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65 not found: ID does not exist" containerID="632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65"
Jul 15 12:37:26.528305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.528243    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65"} err="failed to get container status \"632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65\": rpc error: code = NotFound desc = could not find container \"632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65\": container with ID starting with 632800a5428c84f906e5401a3232c8106eb3766ce869d1686af345a8417d4b65 not found: ID does not exist"
Jul 15 12:37:26.534607 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.534585    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"]
Jul 15 12:37:26.538937 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:26.538913    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ba8973be-ccf667f76-tw2vk"]
Jul 15 12:37:28.062783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:28.062748    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" path="/var/lib/kubelet/pods/d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6/volumes"
Jul 15 12:37:32.659930 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.659896    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"]
Jul 15 12:37:32.660324 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.660308    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" containerName="maas-api"
Jul 15 12:37:32.660324 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.660321    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" containerName="maas-api"
Jul 15 12:37:32.660410 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.660340    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1419b3a-6a83-4fc7-a7e4-0eae612f206a" containerName="revoke-keys"
Jul 15 12:37:32.660410 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.660346    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1419b3a-6a83-4fc7-a7e4-0eae612f206a" containerName="revoke-keys"
Jul 15 12:37:32.660492 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.660418    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1419b3a-6a83-4fc7-a7e4-0eae612f206a" containerName="revoke-keys"
Jul 15 12:37:32.660492 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.660427    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbabcaba-6143-47d8-a63f-d0f5b7362bcb" containerName="revoke-keys"
Jul 15 12:37:32.660492 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.660435    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d048be5d-4776-4e0a-a9b0-65bb2f7fe3d6" containerName="maas-api"
Jul 15 12:37:32.664864 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.664842    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.667156 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.667136    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-078f33f2-serving-cert\""
Jul 15 12:37:32.667334 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.667178    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:37:32.674174 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.674150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"]
Jul 15 12:37:32.781073 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.781031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e6cab4-c61d-47b2-810d-6607346d0d16-openshift-service-ca\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.781267 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.781126    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e6cab4-c61d-47b2-810d-6607346d0d16-maas-api-tls\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.781267 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.781230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qddvq\" (UniqueName: \"kubernetes.io/projected/a2e6cab4-c61d-47b2-810d-6607346d0d16-kube-api-access-qddvq\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.882485 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.882447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e6cab4-c61d-47b2-810d-6607346d0d16-openshift-service-ca\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.882675 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.882532    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e6cab4-c61d-47b2-810d-6607346d0d16-maas-api-tls\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.882729 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.882711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qddvq\" (UniqueName: \"kubernetes.io/projected/a2e6cab4-c61d-47b2-810d-6607346d0d16-kube-api-access-qddvq\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.883173 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.883153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e6cab4-c61d-47b2-810d-6607346d0d16-openshift-service-ca\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.885007 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.884987    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e6cab4-c61d-47b2-810d-6607346d0d16-maas-api-tls\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.890368 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.890344    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qddvq\" (UniqueName: \"kubernetes.io/projected/a2e6cab4-c61d-47b2-810d-6607346d0d16-kube-api-access-qddvq\") pod \"maas-api-e2e-ait-078f33f2-bc9479999-7k77h\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:32.977544 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:32.977498    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:33.122018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:33.121986    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"]
Jul 15 12:37:33.123549 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:37:33.123517    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e6cab4_c61d_47b2_810d_6607346d0d16.slice/crio-eb8b458c5c74995fbef7f81929dcca3e8163ffe348935eece39e57ce3f728a2f WatchSource:0}: Error finding container eb8b458c5c74995fbef7f81929dcca3e8163ffe348935eece39e57ce3f728a2f: Status 404 returned error can't find the container with id eb8b458c5c74995fbef7f81929dcca3e8163ffe348935eece39e57ce3f728a2f
Jul 15 12:37:33.559708 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:33.559669    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h" event={"ID":"a2e6cab4-c61d-47b2-810d-6607346d0d16","Type":"ContainerStarted","Data":"eb8b458c5c74995fbef7f81929dcca3e8163ffe348935eece39e57ce3f728a2f"}
Jul 15 12:37:34.567108 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:34.567055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h" event={"ID":"a2e6cab4-c61d-47b2-810d-6607346d0d16","Type":"ContainerStarted","Data":"47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df"}
Jul 15 12:37:34.567567 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:34.567124    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:34.585931 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:34.585876    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h" podStartSLOduration=2.125535589 podStartE2EDuration="2.585860895s" totalImagesPullingTime="460.325306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:37:34.583270925 +0000 UTC m=+1785.683220289" watchObservedRunningTime="2026-07-15 12:37:34.585860895 +0000 UTC m=+1785.685810232"
Jul 15 12:37:37.689690 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:37.689654    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"]
Jul 15 12:37:37.693310 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:37.693293    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"
Jul 15 12:37:37.695773 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:37.695752    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:37:37.700293 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:37.700273    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"]
Jul 15 12:37:37.825751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:37.825712    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d7cbef-c313-4926-a513-b02c6262383b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-078f33f2-h9mph\" (UID: \"65d7cbef-c313-4926-a513-b02c6262383b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"
Jul 15 12:37:37.927000 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:37.926970    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d7cbef-c313-4926-a513-b02c6262383b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-078f33f2-h9mph\" (UID: \"65d7cbef-c313-4926-a513-b02c6262383b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"
Jul 15 12:37:37.927582 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:37.927562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d7cbef-c313-4926-a513-b02c6262383b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-078f33f2-h9mph\" (UID: \"65d7cbef-c313-4926-a513-b02c6262383b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"
Jul 15 12:37:38.005004 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:38.004973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"
Jul 15 12:37:38.139879 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:38.139851    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"]
Jul 15 12:37:38.140475 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:37:38.140450    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d7cbef_c313_4926_a513_b02c6262383b.slice/crio-315d2db26b2a56014213dd77c0b013adbdb8a3110e5fabb887b53b4ae69e9fae WatchSource:0}: Error finding container 315d2db26b2a56014213dd77c0b013adbdb8a3110e5fabb887b53b4ae69e9fae: Status 404 returned error can't find the container with id 315d2db26b2a56014213dd77c0b013adbdb8a3110e5fabb887b53b4ae69e9fae
Jul 15 12:37:38.591366 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:38.591337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" event={"ID":"65d7cbef-c313-4926-a513-b02c6262383b","Type":"ContainerStarted","Data":"20f4aea797bdbf1e8219c504e5c1731089813b465b786cab9760bb2338f11550"}
Jul 15 12:37:38.591523 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:38.591385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" event={"ID":"65d7cbef-c313-4926-a513-b02c6262383b","Type":"ContainerStarted","Data":"315d2db26b2a56014213dd77c0b013adbdb8a3110e5fabb887b53b4ae69e9fae"}
Jul 15 12:37:38.608646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:38.608562    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" podStartSLOduration=1.608546718 podStartE2EDuration="1.608546718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:37:38.606984223 +0000 UTC m=+1789.706933559" watchObservedRunningTime="2026-07-15 12:37:38.608546718 +0000 UTC m=+1789.708496104"
Jul 15 12:37:39.598310 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:39.598277    2582 generic.go:356] "Generic (PLEG): container finished" podID="65d7cbef-c313-4926-a513-b02c6262383b" containerID="20f4aea797bdbf1e8219c504e5c1731089813b465b786cab9760bb2338f11550" exitCode=7
Jul 15 12:37:39.598721 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:39.598373    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" event={"ID":"65d7cbef-c313-4926-a513-b02c6262383b","Type":"ContainerDied","Data":"20f4aea797bdbf1e8219c504e5c1731089813b465b786cab9760bb2338f11550"}
Jul 15 12:37:39.598721 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:39.598524    2582 scope.go:117] "RemoveContainer" containerID="20f4aea797bdbf1e8219c504e5c1731089813b465b786cab9760bb2338f11550"
Jul 15 12:37:40.578757 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:40.578729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:37:40.605404 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:40.605365    2582 generic.go:356] "Generic (PLEG): container finished" podID="65d7cbef-c313-4926-a513-b02c6262383b" containerID="88f5f101e83532331725bb00e8a8ce10fe8512dcb5bb52a4f2e68d50b6d9b63e" exitCode=7
Jul 15 12:37:40.605865 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:40.605494    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" event={"ID":"65d7cbef-c313-4926-a513-b02c6262383b","Type":"ContainerDied","Data":"88f5f101e83532331725bb00e8a8ce10fe8512dcb5bb52a4f2e68d50b6d9b63e"}
Jul 15 12:37:40.605865 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:40.605551    2582 scope.go:117] "RemoveContainer" containerID="20f4aea797bdbf1e8219c504e5c1731089813b465b786cab9760bb2338f11550"
Jul 15 12:37:40.605865 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:40.605843    2582 scope.go:117] "RemoveContainer" containerID="88f5f101e83532331725bb00e8a8ce10fe8512dcb5bb52a4f2e68d50b6d9b63e"
Jul 15 12:37:40.606138 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:37:40.606111    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-078f33f2-h9mph_odh-ai-gateway-infra(65d7cbef-c313-4926-a513-b02c6262383b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" podUID="65d7cbef-c313-4926-a513-b02c6262383b"
Jul 15 12:37:41.612590 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:41.612503    2582 scope.go:117] "RemoveContainer" containerID="88f5f101e83532331725bb00e8a8ce10fe8512dcb5bb52a4f2e68d50b6d9b63e"
Jul 15 12:37:41.612966 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:37:41.612764    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-078f33f2-h9mph_odh-ai-gateway-infra(65d7cbef-c313-4926-a513-b02c6262383b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" podUID="65d7cbef-c313-4926-a513-b02c6262383b"
Jul 15 12:37:49.530714 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:49.530688    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:37:49.535453 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:49.535432    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:37:53.056671 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:53.056636    2582 scope.go:117] "RemoveContainer" containerID="88f5f101e83532331725bb00e8a8ce10fe8512dcb5bb52a4f2e68d50b6d9b63e"
Jul 15 12:37:53.682479 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:53.682445    2582 generic.go:356] "Generic (PLEG): container finished" podID="65d7cbef-c313-4926-a513-b02c6262383b" containerID="b029e90af05754f16e7f531a263cf19e84808965b26b6e1712e4d47b768fcbc5" exitCode=0
Jul 15 12:37:53.682646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:53.682518    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" event={"ID":"65d7cbef-c313-4926-a513-b02c6262383b","Type":"ContainerDied","Data":"b029e90af05754f16e7f531a263cf19e84808965b26b6e1712e4d47b768fcbc5"}
Jul 15 12:37:53.682646 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:53.682593    2582 scope.go:117] "RemoveContainer" containerID="88f5f101e83532331725bb00e8a8ce10fe8512dcb5bb52a4f2e68d50b6d9b63e"
Jul 15 12:37:54.713289 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:54.713258    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"]
Jul 15 12:37:54.814561 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:54.814540    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"
Jul 15 12:37:54.980672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:54.980643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d7cbef-c313-4926-a513-b02c6262383b-maas-api-service-ca\") pod \"65d7cbef-c313-4926-a513-b02c6262383b\" (UID: \"65d7cbef-c313-4926-a513-b02c6262383b\") "
Jul 15 12:37:54.980986 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:54.980965    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65d7cbef-c313-4926-a513-b02c6262383b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65d7cbef-c313-4926-a513-b02c6262383b" (UID: "65d7cbef-c313-4926-a513-b02c6262383b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:37:55.081899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:55.081869    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d7cbef-c313-4926-a513-b02c6262383b-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:37:55.696305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:55.696274    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"
Jul 15 12:37:55.696492 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:55.696320    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph" event={"ID":"65d7cbef-c313-4926-a513-b02c6262383b","Type":"ContainerDied","Data":"315d2db26b2a56014213dd77c0b013adbdb8a3110e5fabb887b53b4ae69e9fae"}
Jul 15 12:37:55.696492 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:55.696377    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="315d2db26b2a56014213dd77c0b013adbdb8a3110e5fabb887b53b4ae69e9fae"
Jul 15 12:37:55.726782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:55.726751    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"]
Jul 15 12:37:55.730359 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:55.730335    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-h9mph"]
Jul 15 12:37:56.063478 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:37:56.063445    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65d7cbef-c313-4926-a513-b02c6262383b" path="/var/lib/kubelet/pods/65d7cbef-c313-4926-a513-b02c6262383b/volumes"
Jul 15 12:38:27.736856 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.736818    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"]
Jul 15 12:38:27.737456 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.737415    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:27.737456 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.737437    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:27.737583 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.737471    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:27.737583 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.737480    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:27.737691 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.737591    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:27.737691 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.737605    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:27.737691 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.737615    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:27.740773 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.740753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"
Jul 15 12:38:27.742937 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.742920    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:38:27.747812 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.747785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"]
Jul 15 12:38:27.838831 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.838795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cefc82-397d-4bd9-aa22-b3b1f8960d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4\" (UID: \"66cefc82-397d-4bd9-aa22-b3b1f8960d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"
Jul 15 12:38:27.939675 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.939643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cefc82-397d-4bd9-aa22-b3b1f8960d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4\" (UID: \"66cefc82-397d-4bd9-aa22-b3b1f8960d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"
Jul 15 12:38:27.940309 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:27.940290    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cefc82-397d-4bd9-aa22-b3b1f8960d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4\" (UID: \"66cefc82-397d-4bd9-aa22-b3b1f8960d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"
Jul 15 12:38:28.054876 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:28.054790    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"
Jul 15 12:38:28.191343 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:28.191320    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"]
Jul 15 12:38:28.192926 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:38:28.192898    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66cefc82_397d_4bd9_aa22_b3b1f8960d47.slice/crio-ece12b29fa4ab997868e35067caf6e4a1a7dcfd4af7ce98e235164164395f6a7 WatchSource:0}: Error finding container ece12b29fa4ab997868e35067caf6e4a1a7dcfd4af7ce98e235164164395f6a7: Status 404 returned error can't find the container with id ece12b29fa4ab997868e35067caf6e4a1a7dcfd4af7ce98e235164164395f6a7
Jul 15 12:38:28.892715 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:28.892684    2582 generic.go:356] "Generic (PLEG): container finished" podID="66cefc82-397d-4bd9-aa22-b3b1f8960d47" containerID="81e6c648557ec7fe2533c9e8210c32191d35888050f5f1ae5971c2a0640a1d27" exitCode=0
Jul 15 12:38:28.893076 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:28.892773    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4" event={"ID":"66cefc82-397d-4bd9-aa22-b3b1f8960d47","Type":"ContainerDied","Data":"81e6c648557ec7fe2533c9e8210c32191d35888050f5f1ae5971c2a0640a1d27"}
Jul 15 12:38:28.893076 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:28.892820    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4" event={"ID":"66cefc82-397d-4bd9-aa22-b3b1f8960d47","Type":"ContainerStarted","Data":"ece12b29fa4ab997868e35067caf6e4a1a7dcfd4af7ce98e235164164395f6a7"}
Jul 15 12:38:30.034510 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:30.034487    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"
Jul 15 12:38:30.055279 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:30.055252    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cefc82-397d-4bd9-aa22-b3b1f8960d47-maas-api-service-ca\") pod \"66cefc82-397d-4bd9-aa22-b3b1f8960d47\" (UID: \"66cefc82-397d-4bd9-aa22-b3b1f8960d47\") "
Jul 15 12:38:30.055596 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:30.055573    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66cefc82-397d-4bd9-aa22-b3b1f8960d47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66cefc82-397d-4bd9-aa22-b3b1f8960d47" (UID: "66cefc82-397d-4bd9-aa22-b3b1f8960d47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:38:30.156667 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:30.156598    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cefc82-397d-4bd9-aa22-b3b1f8960d47-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:38:30.905268 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:30.905240    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4"
Jul 15 12:38:30.905412 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:30.905283    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4" event={"ID":"66cefc82-397d-4bd9-aa22-b3b1f8960d47","Type":"ContainerDied","Data":"ece12b29fa4ab997868e35067caf6e4a1a7dcfd4af7ce98e235164164395f6a7"}
Jul 15 12:38:30.905412 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:30.905328    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ece12b29fa4ab997868e35067caf6e4a1a7dcfd4af7ce98e235164164395f6a7"
Jul 15 12:38:38.240260 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.240228    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"]
Jul 15 12:38:38.240836 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.240485    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h" podUID="a2e6cab4-c61d-47b2-810d-6607346d0d16" containerName="maas-api" containerID="cri-o://47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df" gracePeriod=30
Jul 15 12:38:38.484725 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.484701    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:38:38.529673 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.529595    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e6cab4-c61d-47b2-810d-6607346d0d16-openshift-service-ca\") pod \"a2e6cab4-c61d-47b2-810d-6607346d0d16\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") "
Jul 15 12:38:38.529673 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.529663    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e6cab4-c61d-47b2-810d-6607346d0d16-maas-api-tls\") pod \"a2e6cab4-c61d-47b2-810d-6607346d0d16\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") "
Jul 15 12:38:38.529877 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.529714    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qddvq\" (UniqueName: \"kubernetes.io/projected/a2e6cab4-c61d-47b2-810d-6607346d0d16-kube-api-access-qddvq\") pod \"a2e6cab4-c61d-47b2-810d-6607346d0d16\" (UID: \"a2e6cab4-c61d-47b2-810d-6607346d0d16\") "
Jul 15 12:38:38.530030 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.530009    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2e6cab4-c61d-47b2-810d-6607346d0d16-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2e6cab4-c61d-47b2-810d-6607346d0d16" (UID: "a2e6cab4-c61d-47b2-810d-6607346d0d16"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:38:38.531832 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.531804    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2e6cab4-c61d-47b2-810d-6607346d0d16-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2e6cab4-c61d-47b2-810d-6607346d0d16" (UID: "a2e6cab4-c61d-47b2-810d-6607346d0d16"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:38:38.531944 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.531835    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e6cab4-c61d-47b2-810d-6607346d0d16-kube-api-access-qddvq" (OuterVolumeSpecName: "kube-api-access-qddvq") pod "a2e6cab4-c61d-47b2-810d-6607346d0d16" (UID: "a2e6cab4-c61d-47b2-810d-6607346d0d16"). InnerVolumeSpecName "kube-api-access-qddvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:38:38.631253 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.631226    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e6cab4-c61d-47b2-810d-6607346d0d16-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:38:38.631253 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.631250    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e6cab4-c61d-47b2-810d-6607346d0d16-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:38:38.631424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.631260    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qddvq\" (UniqueName: \"kubernetes.io/projected/a2e6cab4-c61d-47b2-810d-6607346d0d16-kube-api-access-qddvq\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:38:38.950922 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.950891    2582 generic.go:356] "Generic (PLEG): container finished" podID="a2e6cab4-c61d-47b2-810d-6607346d0d16" containerID="47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df" exitCode=0
Jul 15 12:38:38.951137 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.950934    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"
Jul 15 12:38:38.951137 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.950954    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h" event={"ID":"a2e6cab4-c61d-47b2-810d-6607346d0d16","Type":"ContainerDied","Data":"47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df"}
Jul 15 12:38:38.951137 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.950997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h" event={"ID":"a2e6cab4-c61d-47b2-810d-6607346d0d16","Type":"ContainerDied","Data":"eb8b458c5c74995fbef7f81929dcca3e8163ffe348935eece39e57ce3f728a2f"}
Jul 15 12:38:38.951137 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.951019    2582 scope.go:117] "RemoveContainer" containerID="47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df"
Jul 15 12:38:38.960455 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.960438    2582 scope.go:117] "RemoveContainer" containerID="47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df"
Jul 15 12:38:38.960708 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:38:38.960688    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df\": container with ID starting with 47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df not found: ID does not exist" containerID="47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df"
Jul 15 12:38:38.960763 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.960716    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df"} err="failed to get container status \"47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df\": rpc error: code = NotFound desc = could not find container \"47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df\": container with ID starting with 47819ac5dfe09204e4cf48528bec2f44d1680cd5ae68cab177d18442b0ab16df not found: ID does not exist"
Jul 15 12:38:38.977214 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.977194    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"]
Jul 15 12:38:38.981483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:38.981462    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-078f33f2-bc9479999-7k77h"]
Jul 15 12:38:40.062843 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:40.062809    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2e6cab4-c61d-47b2-810d-6607346d0d16" path="/var/lib/kubelet/pods/a2e6cab4-c61d-47b2-810d-6607346d0d16/volumes"
Jul 15 12:38:55.300648 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.300610    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"]
Jul 15 12:38:55.301250 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301233    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:55.301305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301252    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d7cbef-c313-4926-a513-b02c6262383b" containerName="revoke-keys"
Jul 15 12:38:55.301305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301263    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66cefc82-397d-4bd9-aa22-b3b1f8960d47" containerName="revoke-keys"
Jul 15 12:38:55.301305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301268    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="66cefc82-397d-4bd9-aa22-b3b1f8960d47" containerName="revoke-keys"
Jul 15 12:38:55.301305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301279    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2e6cab4-c61d-47b2-810d-6607346d0d16" containerName="maas-api"
Jul 15 12:38:55.301305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301285    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2e6cab4-c61d-47b2-810d-6607346d0d16" containerName="maas-api"
Jul 15 12:38:55.301484 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301351    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2e6cab4-c61d-47b2-810d-6607346d0d16" containerName="maas-api"
Jul 15 12:38:55.301484 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.301360    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="66cefc82-397d-4bd9-aa22-b3b1f8960d47" containerName="revoke-keys"
Jul 15 12:38:55.304639 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.304620    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.306859 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.306840    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-88ac4482-serving-cert\""
Jul 15 12:38:55.306969 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.306885    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:38:55.316194 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.316175    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"]
Jul 15 12:38:55.385232 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.385199    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.385380 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.385241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jztsx\" (UniqueName: \"kubernetes.io/projected/9c091368-81e6-423f-8b6d-16cb51e1fcd4-kube-api-access-jztsx\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.385380 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.385355    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c091368-81e6-423f-8b6d-16cb51e1fcd4-openshift-service-ca\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.486404 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.486367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c091368-81e6-423f-8b6d-16cb51e1fcd4-openshift-service-ca\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.486553 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.486431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.486553 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.486456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jztsx\" (UniqueName: \"kubernetes.io/projected/9c091368-81e6-423f-8b6d-16cb51e1fcd4-kube-api-access-jztsx\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.486631 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:38:55.486583    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-serving-cert: secret "maas-api-e2e-derive-88ac4482-serving-cert" not found
Jul 15 12:38:55.486688 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:38:55.486673    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls podName:9c091368-81e6-423f-8b6d-16cb51e1fcd4 nodeName:}" failed. No retries permitted until 2026-07-15 12:38:55.986650685 +0000 UTC m=+1867.086600006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls") pod "maas-api-e2e-derive-88ac4482-7787459ccc-8skkt" (UID: "9c091368-81e6-423f-8b6d-16cb51e1fcd4") : secret "maas-api-e2e-derive-88ac4482-serving-cert" not found
Jul 15 12:38:55.487179 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.487163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c091368-81e6-423f-8b6d-16cb51e1fcd4-openshift-service-ca\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.496683 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.496658    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jztsx\" (UniqueName: \"kubernetes.io/projected/9c091368-81e6-423f-8b6d-16cb51e1fcd4-kube-api-access-jztsx\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.990534 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.990499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:55.992857 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:55.992827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls\") pod \"maas-api-e2e-derive-88ac4482-7787459ccc-8skkt\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:56.216418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:56.216385    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:56.570438 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:56.570408    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"]
Jul 15 12:38:56.573044 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:38:56.573013    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c091368_81e6_423f_8b6d_16cb51e1fcd4.slice/crio-10592656d844ce2a0cd6a3e1c004d9fb3fafa1816e79e77a068069763186308a WatchSource:0}: Error finding container 10592656d844ce2a0cd6a3e1c004d9fb3fafa1816e79e77a068069763186308a: Status 404 returned error can't find the container with id 10592656d844ce2a0cd6a3e1c004d9fb3fafa1816e79e77a068069763186308a
Jul 15 12:38:57.061151 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:57.061116    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt" event={"ID":"9c091368-81e6-423f-8b6d-16cb51e1fcd4","Type":"ContainerStarted","Data":"10592656d844ce2a0cd6a3e1c004d9fb3fafa1816e79e77a068069763186308a"}
Jul 15 12:38:58.073387 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:58.073349    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt" event={"ID":"9c091368-81e6-423f-8b6d-16cb51e1fcd4","Type":"ContainerStarted","Data":"0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633"}
Jul 15 12:38:58.073786 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:58.073411    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:38:58.092358 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:58.092309    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt" podStartSLOduration=2.5198524730000003 podStartE2EDuration="3.092294214s" totalImagesPullingTime="572.441741ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:38:58.091364711 +0000 UTC m=+1869.191314080" watchObservedRunningTime="2026-07-15 12:38:58.092294214 +0000 UTC m=+1869.192243552"
Jul 15 12:38:59.623252 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.623214    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"]
Jul 15 12:38:59.627050 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.627030    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"
Jul 15 12:38:59.629261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.629245    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:38:59.634580 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.634556    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"]
Jul 15 12:38:59.723632 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.723600    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adf655a-e1be-4e46-aa74-258238dc77f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7\" (UID: \"4adf655a-e1be-4e46-aa74-258238dc77f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"
Jul 15 12:38:59.824197 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.824156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adf655a-e1be-4e46-aa74-258238dc77f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7\" (UID: \"4adf655a-e1be-4e46-aa74-258238dc77f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"
Jul 15 12:38:59.824794 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.824772    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adf655a-e1be-4e46-aa74-258238dc77f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7\" (UID: \"4adf655a-e1be-4e46-aa74-258238dc77f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"
Jul 15 12:38:59.940152 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:38:59.940042    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"
Jul 15 12:39:00.116703 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:00.116682    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"]
Jul 15 12:39:00.118181 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:39:00.118154    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4adf655a_e1be_4e46_aa74_258238dc77f1.slice/crio-eac25281dc9055d238637927903be1d02974a03378f2581b1d45808d7bdb5aaf WatchSource:0}: Error finding container eac25281dc9055d238637927903be1d02974a03378f2581b1d45808d7bdb5aaf: Status 404 returned error can't find the container with id eac25281dc9055d238637927903be1d02974a03378f2581b1d45808d7bdb5aaf
Jul 15 12:39:01.094783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:01.094687    2582 generic.go:356] "Generic (PLEG): container finished" podID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerID="5254eae694bdf38ef0f9dc2d614b4f98e678c7f059b9a0a354f126eb47a3da87" exitCode=7
Jul 15 12:39:01.094783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:01.094769    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7" event={"ID":"4adf655a-e1be-4e46-aa74-258238dc77f1","Type":"ContainerDied","Data":"5254eae694bdf38ef0f9dc2d614b4f98e678c7f059b9a0a354f126eb47a3da87"}
Jul 15 12:39:01.095254 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:01.094825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7" event={"ID":"4adf655a-e1be-4e46-aa74-258238dc77f1","Type":"ContainerStarted","Data":"eac25281dc9055d238637927903be1d02974a03378f2581b1d45808d7bdb5aaf"}
Jul 15 12:39:01.095254 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:01.094964    2582 scope.go:117] "RemoveContainer" containerID="5254eae694bdf38ef0f9dc2d614b4f98e678c7f059b9a0a354f126eb47a3da87"
Jul 15 12:39:02.102473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:02.102396    2582 generic.go:356] "Generic (PLEG): container finished" podID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerID="b053f1709d1236364beea707dd138278a849c9708d1e0e4f78ec3d8db54e123e" exitCode=7
Jul 15 12:39:02.102473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:02.102456    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7" event={"ID":"4adf655a-e1be-4e46-aa74-258238dc77f1","Type":"ContainerDied","Data":"b053f1709d1236364beea707dd138278a849c9708d1e0e4f78ec3d8db54e123e"}
Jul 15 12:39:02.102899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:02.102495    2582 scope.go:117] "RemoveContainer" containerID="5254eae694bdf38ef0f9dc2d614b4f98e678c7f059b9a0a354f126eb47a3da87"
Jul 15 12:39:02.102899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:02.102658    2582 scope.go:117] "RemoveContainer" containerID="b053f1709d1236364beea707dd138278a849c9708d1e0e4f78ec3d8db54e123e"
Jul 15 12:39:02.103011 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:39:02.102973    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7_odh-ai-gateway-infra(4adf655a-e1be-4e46-aa74-258238dc77f1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1"
Jul 15 12:39:03.109945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:03.109921    2582 scope.go:117] "RemoveContainer" containerID="b053f1709d1236364beea707dd138278a849c9708d1e0e4f78ec3d8db54e123e"
Jul 15 12:39:03.110334 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:39:03.110143    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7_odh-ai-gateway-infra(4adf655a-e1be-4e46-aa74-258238dc77f1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1"
Jul 15 12:39:04.086365 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:04.086332    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:39:16.057080 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:16.057044    2582 scope.go:117] "RemoveContainer" containerID="b053f1709d1236364beea707dd138278a849c9708d1e0e4f78ec3d8db54e123e"
Jul 15 12:39:17.195251 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:17.195163    2582 generic.go:356] "Generic (PLEG): container finished" podID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerID="07cb3d8af32b8d7ec4d4eb24387ec95f7a2e8d7915d9607085ed0cc51cd33c92" exitCode=0
Jul 15 12:39:17.195251 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:17.195227    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7" event={"ID":"4adf655a-e1be-4e46-aa74-258238dc77f1","Type":"ContainerDied","Data":"07cb3d8af32b8d7ec4d4eb24387ec95f7a2e8d7915d9607085ed0cc51cd33c92"}
Jul 15 12:39:17.195661 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:17.195274    2582 scope.go:117] "RemoveContainer" containerID="b053f1709d1236364beea707dd138278a849c9708d1e0e4f78ec3d8db54e123e"
Jul 15 12:39:18.222657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:18.222617    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"]
Jul 15 12:39:18.344676 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:18.344645    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"
Jul 15 12:39:18.393743 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:18.393712    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adf655a-e1be-4e46-aa74-258238dc77f1-maas-api-service-ca\") pod \"4adf655a-e1be-4e46-aa74-258238dc77f1\" (UID: \"4adf655a-e1be-4e46-aa74-258238dc77f1\") "
Jul 15 12:39:18.394069 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:18.394047    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4adf655a-e1be-4e46-aa74-258238dc77f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4adf655a-e1be-4e46-aa74-258238dc77f1" (UID: "4adf655a-e1be-4e46-aa74-258238dc77f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:39:18.494631 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:18.494556    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adf655a-e1be-4e46-aa74-258238dc77f1-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:39:19.212057 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:19.212029    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"
Jul 15 12:39:19.212057 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:19.212028    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7" event={"ID":"4adf655a-e1be-4e46-aa74-258238dc77f1","Type":"ContainerDied","Data":"eac25281dc9055d238637927903be1d02974a03378f2581b1d45808d7bdb5aaf"}
Jul 15 12:39:19.212295 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:19.212111    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eac25281dc9055d238637927903be1d02974a03378f2581b1d45808d7bdb5aaf"
Jul 15 12:39:19.244841 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:19.244811    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"]
Jul 15 12:39:19.256324 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:19.256302    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-h7cf7"]
Jul 15 12:39:20.063740 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:20.063704    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" path="/var/lib/kubelet/pods/4adf655a-e1be-4e46-aa74-258238dc77f1/volumes"
Jul 15 12:39:49.695574 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.695544    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"]
Jul 15 12:39:49.696053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.695937    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:39:49.696053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.695948    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:39:49.696053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.695963    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:39:49.696053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.695969    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:39:49.696053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.696029    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:39:49.696053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.696038    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:39:49.696053 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.696045    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:39:49.699401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.699383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"
Jul 15 12:39:49.701872 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.701847    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:39:49.707622 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.707599    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"]
Jul 15 12:39:49.755154 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.755083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaecb73-d9c3-4b4c-977e-75415ac7c8b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs\" (UID: \"daaecb73-d9c3-4b4c-977e-75415ac7c8b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"
Jul 15 12:39:49.856086 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.856052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaecb73-d9c3-4b4c-977e-75415ac7c8b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs\" (UID: \"daaecb73-d9c3-4b4c-977e-75415ac7c8b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"
Jul 15 12:39:49.856658 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:49.856640    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaecb73-d9c3-4b4c-977e-75415ac7c8b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs\" (UID: \"daaecb73-d9c3-4b4c-977e-75415ac7c8b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"
Jul 15 12:39:50.010690 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:50.010661    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"
Jul 15 12:39:50.148758 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:50.148734    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"]
Jul 15 12:39:50.150869 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:39:50.150843    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaaecb73_d9c3_4b4c_977e_75415ac7c8b8.slice/crio-263d92d9b926db3defcbb732454e82c64fc68b906f0c585612bc59b318aa20b3 WatchSource:0}: Error finding container 263d92d9b926db3defcbb732454e82c64fc68b906f0c585612bc59b318aa20b3: Status 404 returned error can't find the container with id 263d92d9b926db3defcbb732454e82c64fc68b906f0c585612bc59b318aa20b3
Jul 15 12:39:50.397750 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:50.397714    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs" event={"ID":"daaecb73-d9c3-4b4c-977e-75415ac7c8b8","Type":"ContainerStarted","Data":"263d92d9b926db3defcbb732454e82c64fc68b906f0c585612bc59b318aa20b3"}
Jul 15 12:39:51.404987 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:51.404949    2582 generic.go:356] "Generic (PLEG): container finished" podID="daaecb73-d9c3-4b4c-977e-75415ac7c8b8" containerID="5f3f216a7850e4d1aae2b5a526242ba41573c2ebdeb1a0e289a331c6af39c1d7" exitCode=0
Jul 15 12:39:51.405394 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:51.405039    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs" event={"ID":"daaecb73-d9c3-4b4c-977e-75415ac7c8b8","Type":"ContainerDied","Data":"5f3f216a7850e4d1aae2b5a526242ba41573c2ebdeb1a0e289a331c6af39c1d7"}
Jul 15 12:39:52.550069 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:52.550043    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"
Jul 15 12:39:52.682796 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:52.682706    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaecb73-d9c3-4b4c-977e-75415ac7c8b8-maas-api-service-ca\") pod \"daaecb73-d9c3-4b4c-977e-75415ac7c8b8\" (UID: \"daaecb73-d9c3-4b4c-977e-75415ac7c8b8\") "
Jul 15 12:39:52.683073 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:52.683047    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daaecb73-d9c3-4b4c-977e-75415ac7c8b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "daaecb73-d9c3-4b4c-977e-75415ac7c8b8" (UID: "daaecb73-d9c3-4b4c-977e-75415ac7c8b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:39:52.784376 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:52.784340    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaecb73-d9c3-4b4c-977e-75415ac7c8b8-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:39:53.418203 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:53.418174    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs"
Jul 15 12:39:53.418418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:53.418172    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs" event={"ID":"daaecb73-d9c3-4b4c-977e-75415ac7c8b8","Type":"ContainerDied","Data":"263d92d9b926db3defcbb732454e82c64fc68b906f0c585612bc59b318aa20b3"}
Jul 15 12:39:53.418418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:39:53.418297    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="263d92d9b926db3defcbb732454e82c64fc68b906f0c585612bc59b318aa20b3"
Jul 15 12:40:00.152819 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.152791    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"]
Jul 15 12:40:00.155327 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.153032    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt" podUID="9c091368-81e6-423f-8b6d-16cb51e1fcd4" containerName="maas-api" containerID="cri-o://0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633" gracePeriod=30
Jul 15 12:40:00.403563 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.403507    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:40:00.464813 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.464779    2582 generic.go:356] "Generic (PLEG): container finished" podID="9c091368-81e6-423f-8b6d-16cb51e1fcd4" containerID="0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633" exitCode=0
Jul 15 12:40:00.464951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.464820    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"
Jul 15 12:40:00.464951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.464867    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt" event={"ID":"9c091368-81e6-423f-8b6d-16cb51e1fcd4","Type":"ContainerDied","Data":"0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633"}
Jul 15 12:40:00.464951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.464915    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt" event={"ID":"9c091368-81e6-423f-8b6d-16cb51e1fcd4","Type":"ContainerDied","Data":"10592656d844ce2a0cd6a3e1c004d9fb3fafa1816e79e77a068069763186308a"}
Jul 15 12:40:00.464951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.464940    2582 scope.go:117] "RemoveContainer" containerID="0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633"
Jul 15 12:40:00.475022 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.474996    2582 scope.go:117] "RemoveContainer" containerID="0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633"
Jul 15 12:40:00.475300 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:40:00.475278    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633\": container with ID starting with 0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633 not found: ID does not exist" containerID="0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633"
Jul 15 12:40:00.475391 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.475307    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633"} err="failed to get container status \"0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633\": rpc error: code = NotFound desc = could not find container \"0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633\": container with ID starting with 0195c01b7ce0e4532ae3901a1221d5fe10b9a06ba141004af4aa22df8b728633 not found: ID does not exist"
Jul 15 12:40:00.553016 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.552982    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls\") pod \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") "
Jul 15 12:40:00.553202 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.553136    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jztsx\" (UniqueName: \"kubernetes.io/projected/9c091368-81e6-423f-8b6d-16cb51e1fcd4-kube-api-access-jztsx\") pod \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") "
Jul 15 12:40:00.553202 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.553195    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c091368-81e6-423f-8b6d-16cb51e1fcd4-openshift-service-ca\") pod \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\" (UID: \"9c091368-81e6-423f-8b6d-16cb51e1fcd4\") "
Jul 15 12:40:00.553556 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.553528    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c091368-81e6-423f-8b6d-16cb51e1fcd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c091368-81e6-423f-8b6d-16cb51e1fcd4" (UID: "9c091368-81e6-423f-8b6d-16cb51e1fcd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:40:00.555120 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.555080    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c091368-81e6-423f-8b6d-16cb51e1fcd4-kube-api-access-jztsx" (OuterVolumeSpecName: "kube-api-access-jztsx") pod "9c091368-81e6-423f-8b6d-16cb51e1fcd4" (UID: "9c091368-81e6-423f-8b6d-16cb51e1fcd4"). InnerVolumeSpecName "kube-api-access-jztsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:40:00.555217 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.555127    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c091368-81e6-423f-8b6d-16cb51e1fcd4" (UID: "9c091368-81e6-423f-8b6d-16cb51e1fcd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:40:00.654147 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.654067    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c091368-81e6-423f-8b6d-16cb51e1fcd4-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:40:00.654147 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.654118    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jztsx\" (UniqueName: \"kubernetes.io/projected/9c091368-81e6-423f-8b6d-16cb51e1fcd4-kube-api-access-jztsx\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:40:00.654147 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.654129    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c091368-81e6-423f-8b6d-16cb51e1fcd4-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:40:00.786992 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.786959    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"]
Jul 15 12:40:00.791345 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:00.791318    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-88ac4482-7787459ccc-8skkt"]
Jul 15 12:40:02.063489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:40:02.063451    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c091368-81e6-423f-8b6d-16cb51e1fcd4" path="/var/lib/kubelet/pods/9c091368-81e6-423f-8b6d-16cb51e1fcd4/volumes"
Jul 15 12:41:49.672010 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:49.671974    2582 scope.go:117] "RemoveContainer" containerID="02978ccc6cc0c0b717045d219e39596554f8ea650f548772cd5c2c567f2724fb"
Jul 15 12:41:56.476330 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.476296    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"]
Jul 15 12:41:56.476900 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.476880    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:41:56.476984 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.476904    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4adf655a-e1be-4e46-aa74-258238dc77f1" containerName="revoke-keys"
Jul 15 12:41:56.476984 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.476931    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c091368-81e6-423f-8b6d-16cb51e1fcd4" containerName="maas-api"
Jul 15 12:41:56.476984 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.476939    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c091368-81e6-423f-8b6d-16cb51e1fcd4" containerName="maas-api"
Jul 15 12:41:56.476984 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.476963    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daaecb73-d9c3-4b4c-977e-75415ac7c8b8" containerName="revoke-keys"
Jul 15 12:41:56.476984 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.476972    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="daaecb73-d9c3-4b4c-977e-75415ac7c8b8" containerName="revoke-keys"
Jul 15 12:41:56.477271 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.477111    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c091368-81e6-423f-8b6d-16cb51e1fcd4" containerName="maas-api"
Jul 15 12:41:56.477271 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.477129    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="daaecb73-d9c3-4b4c-977e-75415ac7c8b8" containerName="revoke-keys"
Jul 15 12:41:56.480466 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.480446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.482885 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.482863    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-97fdc9d2-serving-cert\""
Jul 15 12:41:56.482982 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.482883    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:41:56.491486 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.491463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"]
Jul 15 12:41:56.594607 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.594568    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h2dv\" (UniqueName: \"kubernetes.io/projected/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-kube-api-access-6h2dv\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.594795 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.594626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-openshift-service-ca\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.594795 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.594680    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.695236 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.695199    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h2dv\" (UniqueName: \"kubernetes.io/projected/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-kube-api-access-6h2dv\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.695399 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.695261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-openshift-service-ca\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.695399 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.695334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.695483 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:41:56.695465    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-serving-cert: secret "maas-api-e2e-mt-97fdc9d2-serving-cert" not found
Jul 15 12:41:56.695560 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:41:56.695549    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls podName:a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d nodeName:}" failed. No retries permitted until 2026-07-15 12:41:57.195528474 +0000 UTC m=+2048.295477811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls") pod "maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv" (UID: "a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d") : secret "maas-api-e2e-mt-97fdc9d2-serving-cert" not found
Jul 15 12:41:56.695877 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.695860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-openshift-service-ca\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:56.705173 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:56.705146    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h2dv\" (UniqueName: \"kubernetes.io/projected/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-kube-api-access-6h2dv\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:57.199361 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:57.199329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:57.201689 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:57.201663    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls\") pod \"maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:57.391862 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:57.391831    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:57.550945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:57.550912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"]
Jul 15 12:41:57.551364 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:41:57.551308    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ef8b71_ba4f_44c0_bf4e_3a9372962a1d.slice/crio-862c1c0c2f5756457224003eab7f565bba795e9cdde688cf51eff13f436bb5dd WatchSource:0}: Error finding container 862c1c0c2f5756457224003eab7f565bba795e9cdde688cf51eff13f436bb5dd: Status 404 returned error can't find the container with id 862c1c0c2f5756457224003eab7f565bba795e9cdde688cf51eff13f436bb5dd
Jul 15 12:41:57.556983 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:57.556963    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:41:58.149807 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:58.149689    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv" event={"ID":"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d","Type":"ContainerStarted","Data":"4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6"}
Jul 15 12:41:58.149807 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:58.149752    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv" event={"ID":"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d","Type":"ContainerStarted","Data":"862c1c0c2f5756457224003eab7f565bba795e9cdde688cf51eff13f436bb5dd"}
Jul 15 12:41:58.149807 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:58.149779    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:41:58.169178 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:41:58.169119    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv" podStartSLOduration=1.8290242110000001 podStartE2EDuration="2.169103431s" totalImagesPullingTime="340.07922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:41:58.166210284 +0000 UTC m=+2049.266159655" watchObservedRunningTime="2026-07-15 12:41:58.169103431 +0000 UTC m=+2049.269052764"
Jul 15 12:42:04.160711 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:04.160679    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:42:09.087229 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.087195    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"]
Jul 15 12:42:09.090924 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.090908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"
Jul 15 12:42:09.093311 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.093290    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:42:09.096797 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.096765    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5abc9a85-c08a-4b26-b878-ee995e97322a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6\" (UID: \"5abc9a85-c08a-4b26-b878-ee995e97322a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"
Jul 15 12:42:09.099080 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.099057    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"]
Jul 15 12:42:09.197405 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.197373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5abc9a85-c08a-4b26-b878-ee995e97322a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6\" (UID: \"5abc9a85-c08a-4b26-b878-ee995e97322a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"
Jul 15 12:42:09.197996 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.197973    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5abc9a85-c08a-4b26-b878-ee995e97322a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6\" (UID: \"5abc9a85-c08a-4b26-b878-ee995e97322a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"
Jul 15 12:42:09.403267 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.403171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"
Jul 15 12:42:09.540122 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:09.540077    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"]
Jul 15 12:42:09.540977 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:42:09.540947    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abc9a85_c08a_4b26_b878_ee995e97322a.slice/crio-d80190b35a52181dbb93617e4e64cb2241b9b78b6a7682ef253cc5e23525afb0 WatchSource:0}: Error finding container d80190b35a52181dbb93617e4e64cb2241b9b78b6a7682ef253cc5e23525afb0: Status 404 returned error can't find the container with id d80190b35a52181dbb93617e4e64cb2241b9b78b6a7682ef253cc5e23525afb0
Jul 15 12:42:10.222204 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:10.222168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6" event={"ID":"5abc9a85-c08a-4b26-b878-ee995e97322a","Type":"ContainerStarted","Data":"60b06ea38e24240654f8a539247ca3f08529f2c12ea3e0c659c9783c8d736ffc"}
Jul 15 12:42:10.222597 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:10.222228    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6" event={"ID":"5abc9a85-c08a-4b26-b878-ee995e97322a","Type":"ContainerStarted","Data":"d80190b35a52181dbb93617e4e64cb2241b9b78b6a7682ef253cc5e23525afb0"}
Jul 15 12:42:10.240783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:10.240736    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6" podStartSLOduration=1.240720494 podStartE2EDuration="1.240720494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:42:10.236601917 +0000 UTC m=+2061.336551254" watchObservedRunningTime="2026-07-15 12:42:10.240720494 +0000 UTC m=+2061.340669884"
Jul 15 12:42:11.229460 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:11.229378    2582 generic.go:356] "Generic (PLEG): container finished" podID="5abc9a85-c08a-4b26-b878-ee995e97322a" containerID="60b06ea38e24240654f8a539247ca3f08529f2c12ea3e0c659c9783c8d736ffc" exitCode=0
Jul 15 12:42:11.229874 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:11.229471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6" event={"ID":"5abc9a85-c08a-4b26-b878-ee995e97322a","Type":"ContainerDied","Data":"60b06ea38e24240654f8a539247ca3f08529f2c12ea3e0c659c9783c8d736ffc"}
Jul 15 12:42:12.367261 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:12.367235    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"
Jul 15 12:42:12.420413 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:12.420368    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5abc9a85-c08a-4b26-b878-ee995e97322a-maas-api-service-ca\") pod \"5abc9a85-c08a-4b26-b878-ee995e97322a\" (UID: \"5abc9a85-c08a-4b26-b878-ee995e97322a\") "
Jul 15 12:42:12.420740 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:12.420721    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5abc9a85-c08a-4b26-b878-ee995e97322a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5abc9a85-c08a-4b26-b878-ee995e97322a" (UID: "5abc9a85-c08a-4b26-b878-ee995e97322a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:42:12.521384 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:12.521302    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5abc9a85-c08a-4b26-b878-ee995e97322a-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:42:13.241374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:13.241345    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6"
Jul 15 12:42:13.241552 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:13.241347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6" event={"ID":"5abc9a85-c08a-4b26-b878-ee995e97322a","Type":"ContainerDied","Data":"d80190b35a52181dbb93617e4e64cb2241b9b78b6a7682ef253cc5e23525afb0"}
Jul 15 12:42:13.241552 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:13.241469    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d80190b35a52181dbb93617e4e64cb2241b9b78b6a7682ef253cc5e23525afb0"
Jul 15 12:42:19.600717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.600679    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"]
Jul 15 12:42:19.601239 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.600946    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv" podUID="a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" containerName="maas-api" containerID="cri-o://4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6" gracePeriod=30
Jul 15 12:42:19.856957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.856900    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:42:19.884237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.884213    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-openshift-service-ca\") pod \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") "
Jul 15 12:42:19.884357 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.884256    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h2dv\" (UniqueName: \"kubernetes.io/projected/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-kube-api-access-6h2dv\") pod \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") "
Jul 15 12:42:19.884397 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.884359    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls\") pod \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\" (UID: \"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d\") "
Jul 15 12:42:19.884582 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.884551    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" (UID: "a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:42:19.884743 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.884726    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:42:19.886268 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.886241    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" (UID: "a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:42:19.886349 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.886327    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-kube-api-access-6h2dv" (OuterVolumeSpecName: "kube-api-access-6h2dv") pod "a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" (UID: "a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d"). InnerVolumeSpecName "kube-api-access-6h2dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:42:19.985426 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.985400    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h2dv\" (UniqueName: \"kubernetes.io/projected/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-kube-api-access-6h2dv\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:42:19.985426 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:19.985424    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:42:20.281356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.281320    2582 generic.go:356] "Generic (PLEG): container finished" podID="a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" containerID="4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6" exitCode=0
Jul 15 12:42:20.281521 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.281365    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"
Jul 15 12:42:20.281521 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.281374    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv" event={"ID":"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d","Type":"ContainerDied","Data":"4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6"}
Jul 15 12:42:20.281521 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.281431    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv" event={"ID":"a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d","Type":"ContainerDied","Data":"862c1c0c2f5756457224003eab7f565bba795e9cdde688cf51eff13f436bb5dd"}
Jul 15 12:42:20.281521 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.281461    2582 scope.go:117] "RemoveContainer" containerID="4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6"
Jul 15 12:42:20.290416 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.290406    2582 scope.go:117] "RemoveContainer" containerID="4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6"
Jul 15 12:42:20.290666 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:42:20.290647    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6\": container with ID starting with 4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6 not found: ID does not exist" containerID="4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6"
Jul 15 12:42:20.290734 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.290676    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6"} err="failed to get container status \"4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6\": rpc error: code = NotFound desc = could not find container \"4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6\": container with ID starting with 4d86d4d1f4b9f056ee0df2bcedc2625730bb9884bcf04b59618667e03513d7a6 not found: ID does not exist"
Jul 15 12:42:20.298527 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.298505    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"]
Jul 15 12:42:20.302766 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:20.302748    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97fdc9d2-66dd796589-jr6sv"]
Jul 15 12:42:22.063156 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:22.063118    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" path="/var/lib/kubelet/pods/a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d/volumes"
Jul 15 12:42:49.566622 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:49.566595    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:42:49.572502 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:49.572482    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:42:49.693941 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:49.693916    2582 scope.go:117] "RemoveContainer" containerID="d8f5ded5ea89d51b6b95a7ac67df8b0dc4353bc04235e765312efef417267e82"
Jul 15 12:42:58.819712 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.819675    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"]
Jul 15 12:42:58.820308 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.820289    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5abc9a85-c08a-4b26-b878-ee995e97322a" containerName="revoke-keys"
Jul 15 12:42:58.820398 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.820311    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5abc9a85-c08a-4b26-b878-ee995e97322a" containerName="revoke-keys"
Jul 15 12:42:58.820398 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.820335    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" containerName="maas-api"
Jul 15 12:42:58.820398 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.820344    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" containerName="maas-api"
Jul 15 12:42:58.820547 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.820437    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9ef8b71-ba4f-44c0-bf4e-3a9372962a1d" containerName="maas-api"
Jul 15 12:42:58.820547 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.820453    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5abc9a85-c08a-4b26-b878-ee995e97322a" containerName="revoke-keys"
Jul 15 12:42:58.824000 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.823975    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:58.826386 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.826363    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:42:58.826386 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.826373    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4f1f2a1d-serving-cert\""
Jul 15 12:42:58.834017 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.833992    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"]
Jul 15 12:42:58.921634 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.921596    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:58.921819 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.921649    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqcbn\" (UniqueName: \"kubernetes.io/projected/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-kube-api-access-mqcbn\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:58.921819 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:58.921696    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-openshift-service-ca\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.022591 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.022553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.022783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.022605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqcbn\" (UniqueName: \"kubernetes.io/projected/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-kube-api-access-mqcbn\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.022783 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.022635    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-openshift-service-ca\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.022783 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:42:59.022718    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-serving-cert: secret "maas-api-e2e-mt-4f1f2a1d-serving-cert" not found
Jul 15 12:42:59.022953 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:42:59.022800    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls podName:11ac8c61-8f96-41e2-88d7-61b15d6bfecf nodeName:}" failed. No retries permitted until 2026-07-15 12:42:59.522779311 +0000 UTC m=+2110.622728635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls") pod "maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8" (UID: "11ac8c61-8f96-41e2-88d7-61b15d6bfecf") : secret "maas-api-e2e-mt-4f1f2a1d-serving-cert" not found
Jul 15 12:42:59.023317 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.023297    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-openshift-service-ca\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.030827 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.030806    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqcbn\" (UniqueName: \"kubernetes.io/projected/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-kube-api-access-mqcbn\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.526462 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.526428    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.528758 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.528736    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls\") pod \"maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:42:59.736268 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:42:59.736230    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:43:00.087390 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:00.087359    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"]
Jul 15 12:43:00.088868 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:43:00.088839    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ac8c61_8f96_41e2_88d7_61b15d6bfecf.slice/crio-5eb61f0b904096f79b9b557765dfab9451e30183596e32d9bace06526865b8b7 WatchSource:0}: Error finding container 5eb61f0b904096f79b9b557765dfab9451e30183596e32d9bace06526865b8b7: Status 404 returned error can't find the container with id 5eb61f0b904096f79b9b557765dfab9451e30183596e32d9bace06526865b8b7
Jul 15 12:43:00.513874 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:00.513832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8" event={"ID":"11ac8c61-8f96-41e2-88d7-61b15d6bfecf","Type":"ContainerStarted","Data":"5eb61f0b904096f79b9b557765dfab9451e30183596e32d9bace06526865b8b7"}
Jul 15 12:43:01.521424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:01.521388    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8" event={"ID":"11ac8c61-8f96-41e2-88d7-61b15d6bfecf","Type":"ContainerStarted","Data":"470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6"}
Jul 15 12:43:01.521790 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:01.521476    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:43:01.537883 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:01.537822    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8" podStartSLOduration=3.017065912 podStartE2EDuration="3.537807147s" totalImagesPullingTime="520.741235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:43:01.537366343 +0000 UTC m=+2112.637315693" watchObservedRunningTime="2026-07-15 12:43:01.537807147 +0000 UTC m=+2112.637756484"
Jul 15 12:43:07.533189 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:07.533157    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:43:41.637123 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.637029    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"]
Jul 15 12:43:41.640978 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.640957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"
Jul 15 12:43:41.643525 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.643505    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:43:41.651599 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.651578    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"]
Jul 15 12:43:41.793362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.793332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f7501e-de95-461d-99a3-d47072be4cd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4\" (UID: \"78f7501e-de95-461d-99a3-d47072be4cd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"
Jul 15 12:43:41.894733 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.894638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f7501e-de95-461d-99a3-d47072be4cd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4\" (UID: \"78f7501e-de95-461d-99a3-d47072be4cd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"
Jul 15 12:43:41.895390 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.895366    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f7501e-de95-461d-99a3-d47072be4cd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4\" (UID: \"78f7501e-de95-461d-99a3-d47072be4cd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"
Jul 15 12:43:41.952835 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:41.952806    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"
Jul 15 12:43:42.088579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:42.088542    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"]
Jul 15 12:43:42.090793 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:43:42.090760    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f7501e_de95_461d_99a3_d47072be4cd5.slice/crio-4ab7664c0eafb246251ab5768afe58ef5504011d62a97ca234af8211df6df271 WatchSource:0}: Error finding container 4ab7664c0eafb246251ab5768afe58ef5504011d62a97ca234af8211df6df271: Status 404 returned error can't find the container with id 4ab7664c0eafb246251ab5768afe58ef5504011d62a97ca234af8211df6df271
Jul 15 12:43:42.768967 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:42.768938    2582 generic.go:356] "Generic (PLEG): container finished" podID="78f7501e-de95-461d-99a3-d47072be4cd5" containerID="d5e8023c44cdb0bf7fad2f5b1b9834092f407ddf9b03eaab1331d73c89fd1057" exitCode=0
Jul 15 12:43:42.769391 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:42.769022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4" event={"ID":"78f7501e-de95-461d-99a3-d47072be4cd5","Type":"ContainerDied","Data":"d5e8023c44cdb0bf7fad2f5b1b9834092f407ddf9b03eaab1331d73c89fd1057"}
Jul 15 12:43:42.769391 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:42.769109    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4" event={"ID":"78f7501e-de95-461d-99a3-d47072be4cd5","Type":"ContainerStarted","Data":"4ab7664c0eafb246251ab5768afe58ef5504011d62a97ca234af8211df6df271"}
Jul 15 12:43:43.911788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:43.911764    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"
Jul 15 12:43:44.015672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:44.015641    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f7501e-de95-461d-99a3-d47072be4cd5-maas-api-service-ca\") pod \"78f7501e-de95-461d-99a3-d47072be4cd5\" (UID: \"78f7501e-de95-461d-99a3-d47072be4cd5\") "
Jul 15 12:43:44.016040 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:44.016018    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78f7501e-de95-461d-99a3-d47072be4cd5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78f7501e-de95-461d-99a3-d47072be4cd5" (UID: "78f7501e-de95-461d-99a3-d47072be4cd5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:43:44.116349 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:44.116313    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f7501e-de95-461d-99a3-d47072be4cd5-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:43:44.781915 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:44.781873    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4"
Jul 15 12:43:44.782143 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:44.781871    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4" event={"ID":"78f7501e-de95-461d-99a3-d47072be4cd5","Type":"ContainerDied","Data":"4ab7664c0eafb246251ab5768afe58ef5504011d62a97ca234af8211df6df271"}
Jul 15 12:43:44.782143 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:44.782033    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ab7664c0eafb246251ab5768afe58ef5504011d62a97ca234af8211df6df271"
Jul 15 12:43:52.076423 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.076393    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"]
Jul 15 12:43:52.076990 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.076666    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8" podUID="11ac8c61-8f96-41e2-88d7-61b15d6bfecf" containerName="maas-api" containerID="cri-o://470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6" gracePeriod=30
Jul 15 12:43:52.327071 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.327011    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:43:52.492225 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.492188    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-openshift-service-ca\") pod \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") "
Jul 15 12:43:52.492434 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.492244    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls\") pod \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") "
Jul 15 12:43:52.492434 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.492301    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqcbn\" (UniqueName: \"kubernetes.io/projected/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-kube-api-access-mqcbn\") pod \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\" (UID: \"11ac8c61-8f96-41e2-88d7-61b15d6bfecf\") "
Jul 15 12:43:52.492625 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.492602    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11ac8c61-8f96-41e2-88d7-61b15d6bfecf" (UID: "11ac8c61-8f96-41e2-88d7-61b15d6bfecf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:43:52.494407 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.494378    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-kube-api-access-mqcbn" (OuterVolumeSpecName: "kube-api-access-mqcbn") pod "11ac8c61-8f96-41e2-88d7-61b15d6bfecf" (UID: "11ac8c61-8f96-41e2-88d7-61b15d6bfecf"). InnerVolumeSpecName "kube-api-access-mqcbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:43:52.494407 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.494388    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11ac8c61-8f96-41e2-88d7-61b15d6bfecf" (UID: "11ac8c61-8f96-41e2-88d7-61b15d6bfecf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:43:52.593943 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.593860    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:43:52.593943 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.593886    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:43:52.593943 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.593895    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqcbn\" (UniqueName: \"kubernetes.io/projected/11ac8c61-8f96-41e2-88d7-61b15d6bfecf-kube-api-access-mqcbn\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:43:52.826280 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.826245    2582 generic.go:356] "Generic (PLEG): container finished" podID="11ac8c61-8f96-41e2-88d7-61b15d6bfecf" containerID="470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6" exitCode=0
Jul 15 12:43:52.826441 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.826291    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"
Jul 15 12:43:52.826441 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.826323    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8" event={"ID":"11ac8c61-8f96-41e2-88d7-61b15d6bfecf","Type":"ContainerDied","Data":"470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6"}
Jul 15 12:43:52.826441 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.826380    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8" event={"ID":"11ac8c61-8f96-41e2-88d7-61b15d6bfecf","Type":"ContainerDied","Data":"5eb61f0b904096f79b9b557765dfab9451e30183596e32d9bace06526865b8b7"}
Jul 15 12:43:52.826441 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.826411    2582 scope.go:117] "RemoveContainer" containerID="470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6"
Jul 15 12:43:52.837230 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.837206    2582 scope.go:117] "RemoveContainer" containerID="470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6"
Jul 15 12:43:52.837547 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:43:52.837523    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6\": container with ID starting with 470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6 not found: ID does not exist" containerID="470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6"
Jul 15 12:43:52.837621 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.837559    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6"} err="failed to get container status \"470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6\": rpc error: code = NotFound desc = could not find container \"470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6\": container with ID starting with 470a627342e4d0d75d290d1867e7a562e0e2568c3ecf1762f8eef97c896ce8d6 not found: ID does not exist"
Jul 15 12:43:52.849549 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.849476    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"]
Jul 15 12:43:52.852059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:52.852033    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f1f2a1d-567857b664-gpxc8"]
Jul 15 12:43:54.063656 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:43:54.063624    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11ac8c61-8f96-41e2-88d7-61b15d6bfecf" path="/var/lib/kubelet/pods/11ac8c61-8f96-41e2-88d7-61b15d6bfecf/volumes"
Jul 15 12:44:49.726299 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:44:49.726268    2582 scope.go:117] "RemoveContainer" containerID="b029e90af05754f16e7f531a263cf19e84808965b26b6e1712e4d47b768fcbc5"
Jul 15 12:45:00.149084 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.149047    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx"]
Jul 15 12:45:00.149476 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.149463    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ac8c61-8f96-41e2-88d7-61b15d6bfecf" containerName="maas-api"
Jul 15 12:45:00.149476 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.149477    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ac8c61-8f96-41e2-88d7-61b15d6bfecf" containerName="maas-api"
Jul 15 12:45:00.149548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.149493    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f7501e-de95-461d-99a3-d47072be4cd5" containerName="revoke-keys"
Jul 15 12:45:00.149548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.149499    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f7501e-de95-461d-99a3-d47072be4cd5" containerName="revoke-keys"
Jul 15 12:45:00.149613 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.149558    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="78f7501e-de95-461d-99a3-d47072be4cd5" containerName="revoke-keys"
Jul 15 12:45:00.149613 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.149567    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="11ac8c61-8f96-41e2-88d7-61b15d6bfecf" containerName="maas-api"
Jul 15 12:45:00.152657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.152639    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx"
Jul 15 12:45:00.155455 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.155364    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:45:00.161323 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.160637    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx"]
Jul 15 12:45:00.164479 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.164460    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx"
Jul 15 12:45:00.308536 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:00.308503    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx"]
Jul 15 12:45:01.222103 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:01.222049    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx" event={"ID":"5ea6d91d-d68b-40fc-9f52-a8052d23cda6","Type":"ContainerStarted","Data":"2f01ca41542674636fbe82620a813a2b57397005d6c9136d82b8686ed85034eb"}
Jul 15 12:45:01.222556 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:01.222130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx" event={"ID":"5ea6d91d-d68b-40fc-9f52-a8052d23cda6","Type":"ContainerStarted","Data":"d10caca347727fcb6ffc6c93af6b7e1e3483b842b7faad64622056b1256d2040"}
Jul 15 12:45:01.241809 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:01.241751    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx" podStartSLOduration=1.2417332700000001 podStartE2EDuration="1.24173327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:45:01.240032281 +0000 UTC m=+2232.339981641" watchObservedRunningTime="2026-07-15 12:45:01.24173327 +0000 UTC m=+2232.341682608"
Jul 15 12:45:02.228799 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:02.228762    2582 generic.go:356] "Generic (PLEG): container finished" podID="5ea6d91d-d68b-40fc-9f52-a8052d23cda6" containerID="2f01ca41542674636fbe82620a813a2b57397005d6c9136d82b8686ed85034eb" exitCode=0
Jul 15 12:45:02.229237 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:02.228845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx" event={"ID":"5ea6d91d-d68b-40fc-9f52-a8052d23cda6","Type":"ContainerDied","Data":"2f01ca41542674636fbe82620a813a2b57397005d6c9136d82b8686ed85034eb"}
Jul 15 12:45:03.371823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:03.371795    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx"
Jul 15 12:45:04.243918 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:04.243888    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx"
Jul 15 12:45:04.243918 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:04.243900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-qmvzx" event={"ID":"5ea6d91d-d68b-40fc-9f52-a8052d23cda6","Type":"ContainerDied","Data":"d10caca347727fcb6ffc6c93af6b7e1e3483b842b7faad64622056b1256d2040"}
Jul 15 12:45:04.244196 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:04.243953    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d10caca347727fcb6ffc6c93af6b7e1e3483b842b7faad64622056b1256d2040"
Jul 15 12:45:49.755394 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:45:49.755361    2582 scope.go:117] "RemoveContainer" containerID="07cb3d8af32b8d7ec4d4eb24387ec95f7a2e8d7915d9607085ed0cc51cd33c92"
Jul 15 12:46:00.844583 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.844548    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"]
Jul 15 12:46:00.851837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.849359    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea6d91d-d68b-40fc-9f52-a8052d23cda6" containerName="cleanup"
Jul 15 12:46:00.851837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.849445    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea6d91d-d68b-40fc-9f52-a8052d23cda6" containerName="cleanup"
Jul 15 12:46:00.851837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.849719    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea6d91d-d68b-40fc-9f52-a8052d23cda6" containerName="cleanup"
Jul 15 12:46:00.854205 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.854176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:00.856531 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.856492    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fec782-serving-cert\""
Jul 15 12:46:00.856833 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.856561    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:46:00.859321 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.859298    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"]
Jul 15 12:46:00.913595 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.913554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:00.913824 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.913620    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr9ld\" (UniqueName: \"kubernetes.io/projected/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-kube-api-access-rr9ld\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:00.913824 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:00.913733    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.014399 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.014358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.014623 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.014409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr9ld\" (UniqueName: \"kubernetes.io/projected/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-kube-api-access-rr9ld\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.014623 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.014447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.014623 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:46:01.014489    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-serving-cert: secret "maas-api-e2e-inf-a-fec782-serving-cert" not found
Jul 15 12:46:01.014623 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:46:01.014571    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls podName:d14e9bda-32e5-44f3-8c8f-79d5f0eb437b nodeName:}" failed. No retries permitted until 2026-07-15 12:46:01.514551912 +0000 UTC m=+2292.614501237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls") pod "maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz" (UID: "d14e9bda-32e5-44f3-8c8f-79d5f0eb437b") : secret "maas-api-e2e-inf-a-fec782-serving-cert" not found
Jul 15 12:46:01.015178 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.015159    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.023753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.023728    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr9ld\" (UniqueName: \"kubernetes.io/projected/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-kube-api-access-rr9ld\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.519520 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.519479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.521844 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.521818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls\") pod \"maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.767975 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.767947    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:01.912034 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:01.912003    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"]
Jul 15 12:46:01.913261 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:46:01.913233    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14e9bda_32e5_44f3_8c8f_79d5f0eb437b.slice/crio-978fdbb76154e51c40c582c525564acbe02fbee093ad500296d0d3d4897e33c0 WatchSource:0}: Error finding container 978fdbb76154e51c40c582c525564acbe02fbee093ad500296d0d3d4897e33c0: Status 404 returned error can't find the container with id 978fdbb76154e51c40c582c525564acbe02fbee093ad500296d0d3d4897e33c0
Jul 15 12:46:02.577781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:02.577748    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz" event={"ID":"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b","Type":"ContainerStarted","Data":"f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061"}
Jul 15 12:46:02.577781 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:02.577804    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz" event={"ID":"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b","Type":"ContainerStarted","Data":"978fdbb76154e51c40c582c525564acbe02fbee093ad500296d0d3d4897e33c0"}
Jul 15 12:46:02.578056 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:02.577918    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:02.602214 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:02.602159    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz" podStartSLOduration=2.106787987 podStartE2EDuration="2.60214337s" totalImagesPullingTime="495.355383ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:46:02.59995389 +0000 UTC m=+2293.699903251" watchObservedRunningTime="2026-07-15 12:46:02.60214337 +0000 UTC m=+2293.702092707"
Jul 15 12:46:08.588714 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.588682    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:46:08.818489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.818459    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"]
Jul 15 12:46:08.822321 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.822296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.824747 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.824725    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-93edea-serving-cert\""
Jul 15 12:46:08.832107 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.832073    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"]
Jul 15 12:46:08.882450 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.882365    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dgw8\" (UniqueName: \"kubernetes.io/projected/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-kube-api-access-5dgw8\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.882450 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.882411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.882450 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.882438    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.983688 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.983648    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dgw8\" (UniqueName: \"kubernetes.io/projected/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-kube-api-access-5dgw8\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.983860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.983706    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.983860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.983739    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.983933 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:46:08.983866    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-serving-cert: secret "maas-api-e2e-inf-b-93edea-serving-cert" not found
Jul 15 12:46:08.983933 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:46:08.983931    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls podName:7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6 nodeName:}" failed. No retries permitted until 2026-07-15 12:46:09.483911778 +0000 UTC m=+2300.583861094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls") pod "maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj" (UID: "7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6") : secret "maas-api-e2e-inf-b-93edea-serving-cert" not found
Jul 15 12:46:08.984394 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.984375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:08.994767 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:08.994740    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dgw8\" (UniqueName: \"kubernetes.io/projected/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-kube-api-access-5dgw8\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:09.489560 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:09.489522    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:09.491910 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:09.491884    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls\") pod \"maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:09.733974 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:09.733942    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:09.879152 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:09.879115    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"]
Jul 15 12:46:09.880601 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:46:09.880573    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d2fa191_f3a9_4a9a_85a1_7dc9ef59ddd6.slice/crio-d93122a066656fd3d89da2a258964dfb4df28c021e16c6768ffe9a0f504853e5 WatchSource:0}: Error finding container d93122a066656fd3d89da2a258964dfb4df28c021e16c6768ffe9a0f504853e5: Status 404 returned error can't find the container with id d93122a066656fd3d89da2a258964dfb4df28c021e16c6768ffe9a0f504853e5
Jul 15 12:46:10.628616 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:10.628512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj" event={"ID":"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6","Type":"ContainerStarted","Data":"2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d"}
Jul 15 12:46:10.628616 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:10.628573    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj" event={"ID":"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6","Type":"ContainerStarted","Data":"d93122a066656fd3d89da2a258964dfb4df28c021e16c6768ffe9a0f504853e5"}
Jul 15 12:46:10.628616 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:10.628597    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:46:10.653928 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:10.653868    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj" podStartSLOduration=2.262386603 podStartE2EDuration="2.653852359s" totalImagesPullingTime="391.465756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:46:10.650991541 +0000 UTC m=+2301.750940878" watchObservedRunningTime="2026-07-15 12:46:10.653852359 +0000 UTC m=+2301.753801696"
Jul 15 12:46:16.640704 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:46:16.640672    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:47:28.778509 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:28.778474    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"]
Jul 15 12:47:28.782018 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:28.781999    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"
Jul 15 12:47:28.787541 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:28.787521    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:47:28.792852 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:28.792826    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"]
Jul 15 12:47:28.897754 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:28.897719    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40ec829-e7de-4325-988a-b3bff5561e10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z\" (UID: \"e40ec829-e7de-4325-988a-b3bff5561e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"
Jul 15 12:47:28.999052 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:28.999020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40ec829-e7de-4325-988a-b3bff5561e10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z\" (UID: \"e40ec829-e7de-4325-988a-b3bff5561e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"
Jul 15 12:47:28.999624 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:28.999605    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40ec829-e7de-4325-988a-b3bff5561e10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z\" (UID: \"e40ec829-e7de-4325-988a-b3bff5561e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"
Jul 15 12:47:29.092966 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:29.092890    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"
Jul 15 12:47:29.230132 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:29.230088    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"]
Jul 15 12:47:29.231427 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:47:29.231391    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40ec829_e7de_4325_988a_b3bff5561e10.slice/crio-4a78dd503701909286f14d6b69d6008549d07581a8e69ceee1c423ebf24c0035 WatchSource:0}: Error finding container 4a78dd503701909286f14d6b69d6008549d07581a8e69ceee1c423ebf24c0035: Status 404 returned error can't find the container with id 4a78dd503701909286f14d6b69d6008549d07581a8e69ceee1c423ebf24c0035
Jul 15 12:47:30.092894 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:30.092858    2582 generic.go:356] "Generic (PLEG): container finished" podID="e40ec829-e7de-4325-988a-b3bff5561e10" containerID="35a6eb3a349f900f461245b5fd5e5fe3329851521803e766290eeec9bcdb046c" exitCode=0
Jul 15 12:47:30.093287 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:30.092910    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z" event={"ID":"e40ec829-e7de-4325-988a-b3bff5561e10","Type":"ContainerDied","Data":"35a6eb3a349f900f461245b5fd5e5fe3329851521803e766290eeec9bcdb046c"}
Jul 15 12:47:30.093287 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:30.092971    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z" event={"ID":"e40ec829-e7de-4325-988a-b3bff5561e10","Type":"ContainerStarted","Data":"4a78dd503701909286f14d6b69d6008549d07581a8e69ceee1c423ebf24c0035"}
Jul 15 12:47:31.233978 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:31.233954    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"
Jul 15 12:47:31.320444 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:31.320405    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40ec829-e7de-4325-988a-b3bff5561e10-maas-api-service-ca\") pod \"e40ec829-e7de-4325-988a-b3bff5561e10\" (UID: \"e40ec829-e7de-4325-988a-b3bff5561e10\") "
Jul 15 12:47:31.320762 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:31.320737    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e40ec829-e7de-4325-988a-b3bff5561e10-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e40ec829-e7de-4325-988a-b3bff5561e10" (UID: "e40ec829-e7de-4325-988a-b3bff5561e10"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:47:31.422004 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:31.421919    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40ec829-e7de-4325-988a-b3bff5561e10-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:47:32.106996 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:32.106221    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z" event={"ID":"e40ec829-e7de-4325-988a-b3bff5561e10","Type":"ContainerDied","Data":"4a78dd503701909286f14d6b69d6008549d07581a8e69ceee1c423ebf24c0035"}
Jul 15 12:47:32.106996 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:32.106282    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a78dd503701909286f14d6b69d6008549d07581a8e69ceee1c423ebf24c0035"
Jul 15 12:47:32.106996 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:32.106390    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z"
Jul 15 12:47:39.489105 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.489055    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"]
Jul 15 12:47:39.489625 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.489306    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz" podUID="d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" containerName="maas-api" containerID="cri-o://f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061" gracePeriod=30
Jul 15 12:47:39.743166 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.743072    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:47:39.900022 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.899988    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-openshift-service-ca\") pod \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") "
Jul 15 12:47:39.900196 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.900069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls\") pod \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") "
Jul 15 12:47:39.900196 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.900136    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr9ld\" (UniqueName: \"kubernetes.io/projected/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-kube-api-access-rr9ld\") pod \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\" (UID: \"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b\") "
Jul 15 12:47:39.900426 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.900395    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" (UID: "d14e9bda-32e5-44f3-8c8f-79d5f0eb437b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:47:39.901994 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.901977    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" (UID: "d14e9bda-32e5-44f3-8c8f-79d5f0eb437b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:47:39.902103 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:39.902077    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-kube-api-access-rr9ld" (OuterVolumeSpecName: "kube-api-access-rr9ld") pod "d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" (UID: "d14e9bda-32e5-44f3-8c8f-79d5f0eb437b"). InnerVolumeSpecName "kube-api-access-rr9ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:47:40.001519 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.001445    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:47:40.001519 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.001474    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr9ld\" (UniqueName: \"kubernetes.io/projected/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-kube-api-access-rr9ld\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:47:40.001519 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.001485    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:47:40.156348 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.156310    2582 generic.go:356] "Generic (PLEG): container finished" podID="d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" containerID="f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061" exitCode=0
Jul 15 12:47:40.156513 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.156359    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"
Jul 15 12:47:40.156513 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.156394    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz" event={"ID":"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b","Type":"ContainerDied","Data":"f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061"}
Jul 15 12:47:40.156513 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.156448    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz" event={"ID":"d14e9bda-32e5-44f3-8c8f-79d5f0eb437b","Type":"ContainerDied","Data":"978fdbb76154e51c40c582c525564acbe02fbee093ad500296d0d3d4897e33c0"}
Jul 15 12:47:40.156513 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.156473    2582 scope.go:117] "RemoveContainer" containerID="f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061"
Jul 15 12:47:40.165531 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.165514    2582 scope.go:117] "RemoveContainer" containerID="f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061"
Jul 15 12:47:40.165795 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:47:40.165774    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061\": container with ID starting with f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061 not found: ID does not exist" containerID="f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061"
Jul 15 12:47:40.165873 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.165807    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061"} err="failed to get container status \"f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061\": rpc error: code = NotFound desc = could not find container \"f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061\": container with ID starting with f4a20341bb128fe4e7cbe82bc8b3d26e5987e508b7651a89d342089aa73ef061 not found: ID does not exist"
Jul 15 12:47:40.179199 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.179177    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"]
Jul 15 12:47:40.186121 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:40.186089    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fec782-7f8d8c8fd6-cjdnz"]
Jul 15 12:47:42.063060 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:42.063026    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" path="/var/lib/kubelet/pods/d14e9bda-32e5-44f3-8c8f-79d5f0eb437b/volumes"
Jul 15 12:47:49.603668 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:49.603640    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:47:49.610384 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:49.610363    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:47:55.022951 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.022917    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"]
Jul 15 12:47:55.023356 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.023342    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e40ec829-e7de-4325-988a-b3bff5561e10" containerName="revoke-keys"
Jul 15 12:47:55.023401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.023357    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e40ec829-e7de-4325-988a-b3bff5561e10" containerName="revoke-keys"
Jul 15 12:47:55.023401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.023370    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" containerName="maas-api"
Jul 15 12:47:55.023401 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.023376    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" containerName="maas-api"
Jul 15 12:47:55.023491 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.023438    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e40ec829-e7de-4325-988a-b3bff5561e10" containerName="revoke-keys"
Jul 15 12:47:55.023491 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.023445    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d14e9bda-32e5-44f3-8c8f-79d5f0eb437b" containerName="maas-api"
Jul 15 12:47:55.028076 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.028059    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"
Jul 15 12:47:55.030851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.030828    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:47:55.038959 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.038935    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"]
Jul 15 12:47:55.131424 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.131389    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cc217a-8069-4a2b-a406-c0a23c8e9ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm\" (UID: \"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"
Jul 15 12:47:55.232346 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.232306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cc217a-8069-4a2b-a406-c0a23c8e9ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm\" (UID: \"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"
Jul 15 12:47:55.233062 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.233043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cc217a-8069-4a2b-a406-c0a23c8e9ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm\" (UID: \"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"
Jul 15 12:47:55.339698 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.339600    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"
Jul 15 12:47:55.482649 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:55.482615    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"]
Jul 15 12:47:55.483600 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:47:55.483577    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cc217a_8069_4a2b_a406_c0a23c8e9ba2.slice/crio-7aa3403d6ec75aa3a0f34fe6e3d328eba85ee122d0ffe3b2fbd1c2e0a188c47d WatchSource:0}: Error finding container 7aa3403d6ec75aa3a0f34fe6e3d328eba85ee122d0ffe3b2fbd1c2e0a188c47d: Status 404 returned error can't find the container with id 7aa3403d6ec75aa3a0f34fe6e3d328eba85ee122d0ffe3b2fbd1c2e0a188c47d
Jul 15 12:47:56.256920 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:56.256883    2582 generic.go:356] "Generic (PLEG): container finished" podID="f8cc217a-8069-4a2b-a406-c0a23c8e9ba2" containerID="5cc2f68d541bdf559e7b14a8dfb9d83eb12ad596178f07a3c6224cbee22549db" exitCode=0
Jul 15 12:47:56.257358 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:56.256931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm" event={"ID":"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2","Type":"ContainerDied","Data":"5cc2f68d541bdf559e7b14a8dfb9d83eb12ad596178f07a3c6224cbee22549db"}
Jul 15 12:47:56.257358 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:56.256997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm" event={"ID":"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2","Type":"ContainerStarted","Data":"7aa3403d6ec75aa3a0f34fe6e3d328eba85ee122d0ffe3b2fbd1c2e0a188c47d"}
Jul 15 12:47:57.397748 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:57.397728    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"
Jul 15 12:47:57.451550 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:57.451521    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cc217a-8069-4a2b-a406-c0a23c8e9ba2-maas-api-service-ca\") pod \"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2\" (UID: \"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2\") "
Jul 15 12:47:57.451865 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:57.451842    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8cc217a-8069-4a2b-a406-c0a23c8e9ba2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8cc217a-8069-4a2b-a406-c0a23c8e9ba2" (UID: "f8cc217a-8069-4a2b-a406-c0a23c8e9ba2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:47:57.552857 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:57.552781    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cc217a-8069-4a2b-a406-c0a23c8e9ba2-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:47:58.270505 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:58.270477    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm"
Jul 15 12:47:58.270667 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:58.270506    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm" event={"ID":"f8cc217a-8069-4a2b-a406-c0a23c8e9ba2","Type":"ContainerDied","Data":"7aa3403d6ec75aa3a0f34fe6e3d328eba85ee122d0ffe3b2fbd1c2e0a188c47d"}
Jul 15 12:47:58.270667 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:47:58.270550    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7aa3403d6ec75aa3a0f34fe6e3d328eba85ee122d0ffe3b2fbd1c2e0a188c47d"
Jul 15 12:48:05.912532 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:05.912496    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"]
Jul 15 12:48:05.912945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:05.912791    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj" podUID="7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" containerName="maas-api" containerID="cri-o://2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d" gracePeriod=30
Jul 15 12:48:06.173691 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.173631    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:48:06.227770 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.227729    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-openshift-service-ca\") pod \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") "
Jul 15 12:48:06.227944 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.227874    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dgw8\" (UniqueName: \"kubernetes.io/projected/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-kube-api-access-5dgw8\") pod \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") "
Jul 15 12:48:06.227944 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.227902    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls\") pod \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\" (UID: \"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6\") "
Jul 15 12:48:06.228127 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.228080    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" (UID: "7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:48:06.228230 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.228212    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:48:06.230008 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.229984    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-kube-api-access-5dgw8" (OuterVolumeSpecName: "kube-api-access-5dgw8") pod "7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" (UID: "7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6"). InnerVolumeSpecName "kube-api-access-5dgw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:48:06.230122 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.230058    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" (UID: "7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:48:06.317622 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.317583    2582 generic.go:356] "Generic (PLEG): container finished" podID="7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" containerID="2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d" exitCode=0
Jul 15 12:48:06.317761 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.317634    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"
Jul 15 12:48:06.317761 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.317670    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj" event={"ID":"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6","Type":"ContainerDied","Data":"2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d"}
Jul 15 12:48:06.317761 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.317726    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj" event={"ID":"7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6","Type":"ContainerDied","Data":"d93122a066656fd3d89da2a258964dfb4df28c021e16c6768ffe9a0f504853e5"}
Jul 15 12:48:06.317761 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.317749    2582 scope.go:117] "RemoveContainer" containerID="2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d"
Jul 15 12:48:06.327299 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.327241    2582 scope.go:117] "RemoveContainer" containerID="2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d"
Jul 15 12:48:06.327526 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:48:06.327506    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d\": container with ID starting with 2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d not found: ID does not exist" containerID="2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d"
Jul 15 12:48:06.327570 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.327536    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d"} err="failed to get container status \"2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d\": rpc error: code = NotFound desc = could not find container \"2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d\": container with ID starting with 2d8061deb6f035e527395c296d8e6dc9beeb5c9e05b13da14ebd0e3ad299913d not found: ID does not exist"
Jul 15 12:48:06.328974 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.328957    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dgw8\" (UniqueName: \"kubernetes.io/projected/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-kube-api-access-5dgw8\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:48:06.329023 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.328977    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:48:06.340064 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.340033    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"]
Jul 15 12:48:06.346438 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:06.346407    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-93edea-d79654dbc-5ndvj"]
Jul 15 12:48:08.063232 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:08.063200    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" path="/var/lib/kubelet/pods/7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6/volumes"
Jul 15 12:48:25.809900 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.809866    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"]
Jul 15 12:48:25.810369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.810266    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8cc217a-8069-4a2b-a406-c0a23c8e9ba2" containerName="revoke-keys"
Jul 15 12:48:25.810369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.810277    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8cc217a-8069-4a2b-a406-c0a23c8e9ba2" containerName="revoke-keys"
Jul 15 12:48:25.810369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.810299    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" containerName="maas-api"
Jul 15 12:48:25.810369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.810305    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" containerName="maas-api"
Jul 15 12:48:25.810369 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.810368    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8cc217a-8069-4a2b-a406-c0a23c8e9ba2" containerName="revoke-keys"
Jul 15 12:48:25.810526 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.810377    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d2fa191-f3a9-4a9a-85a1-7dc9ef59ddd6" containerName="maas-api"
Jul 15 12:48:25.813549 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.813528    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:25.816807 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.816782    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:48:25.817012 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.816996    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a77f45-serving-cert\""
Jul 15 12:48:25.854601 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.854574    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"]
Jul 15 12:48:25.907691 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.907644    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:25.907851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.907729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99jd9\" (UniqueName: \"kubernetes.io/projected/f9f13e3f-f538-4611-8cc7-ad0937f86a44-kube-api-access-99jd9\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:25.907851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:25.907766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f13e3f-f538-4611-8cc7-ad0937f86a44-openshift-service-ca\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.008563 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.008527    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99jd9\" (UniqueName: \"kubernetes.io/projected/f9f13e3f-f538-4611-8cc7-ad0937f86a44-kube-api-access-99jd9\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.008727 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.008578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f13e3f-f538-4611-8cc7-ad0937f86a44-openshift-service-ca\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.008727 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.008638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.008817 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:48:26.008754    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-serving-cert: secret "maas-api-e2e-api-a-a77f45-serving-cert" not found
Jul 15 12:48:26.008854 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:48:26.008831    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls podName:f9f13e3f-f538-4611-8cc7-ad0937f86a44 nodeName:}" failed. No retries permitted until 2026-07-15 12:48:26.508810829 +0000 UTC m=+2437.608760146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls") pod "maas-api-e2e-api-a-a77f45-58654568d4-7j455" (UID: "f9f13e3f-f538-4611-8cc7-ad0937f86a44") : secret "maas-api-e2e-api-a-a77f45-serving-cert" not found
Jul 15 12:48:26.009417 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.009396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f13e3f-f538-4611-8cc7-ad0937f86a44-openshift-service-ca\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.023027 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.022999    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99jd9\" (UniqueName: \"kubernetes.io/projected/f9f13e3f-f538-4611-8cc7-ad0937f86a44-kube-api-access-99jd9\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.513521 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.513489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.515964 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.515934    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls\") pod \"maas-api-e2e-api-a-a77f45-58654568d4-7j455\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:26.723737 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:26.723700    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:27.073686 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:27.073654    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"]
Jul 15 12:48:27.075270 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:48:27.075230    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f13e3f_f538_4611_8cc7_ad0937f86a44.slice/crio-e289abd7a4a285325171e3fd398f45300b3284fbbf88aa3c2e2373274e2a6c29 WatchSource:0}: Error finding container e289abd7a4a285325171e3fd398f45300b3284fbbf88aa3c2e2373274e2a6c29: Status 404 returned error can't find the container with id e289abd7a4a285325171e3fd398f45300b3284fbbf88aa3c2e2373274e2a6c29
Jul 15 12:48:27.076427 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:27.076407    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:48:27.455761 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:27.455725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455" event={"ID":"f9f13e3f-f538-4611-8cc7-ad0937f86a44","Type":"ContainerStarted","Data":"e289abd7a4a285325171e3fd398f45300b3284fbbf88aa3c2e2373274e2a6c29"}
Jul 15 12:48:28.465459 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:28.465423    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455" event={"ID":"f9f13e3f-f538-4611-8cc7-ad0937f86a44","Type":"ContainerStarted","Data":"9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557"}
Jul 15 12:48:28.465853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:28.465586    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:28.486851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:28.486803    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455" podStartSLOduration=2.435636277 podStartE2EDuration="3.486787388s" totalImagesPullingTime="1.051151111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:48:28.484969502 +0000 UTC m=+2439.584918840" watchObservedRunningTime="2026-07-15 12:48:28.486787388 +0000 UTC m=+2439.586736725"
Jul 15 12:48:33.729516 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.729481    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"]
Jul 15 12:48:33.736374 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.736349    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.740813 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.740790    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a2a9e1-serving-cert\""
Jul 15 12:48:33.746275 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.746245    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"]
Jul 15 12:48:33.885790 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.885757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.885972 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.885812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24hjs\" (UniqueName: \"kubernetes.io/projected/13fda6cb-7928-403e-a014-843a09be2eda-kube-api-access-24hjs\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.885972 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.885920    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fda6cb-7928-403e-a014-843a09be2eda-openshift-service-ca\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.987171 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.987071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.987171 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.987144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24hjs\" (UniqueName: \"kubernetes.io/projected/13fda6cb-7928-403e-a014-843a09be2eda-kube-api-access-24hjs\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.987366 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.987200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fda6cb-7928-403e-a014-843a09be2eda-openshift-service-ca\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.987366 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:48:33.987229    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-serving-cert: secret "maas-api-e2e-api-b-a2a9e1-serving-cert" not found
Jul 15 12:48:33.987366 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:48:33.987297    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls podName:13fda6cb-7928-403e-a014-843a09be2eda nodeName:}" failed. No retries permitted until 2026-07-15 12:48:34.487279312 +0000 UTC m=+2445.587228628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls") pod "maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9" (UID: "13fda6cb-7928-403e-a014-843a09be2eda") : secret "maas-api-e2e-api-b-a2a9e1-serving-cert" not found
Jul 15 12:48:33.987760 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.987743    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fda6cb-7928-403e-a014-843a09be2eda-openshift-service-ca\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:33.999498 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:33.999470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24hjs\" (UniqueName: \"kubernetes.io/projected/13fda6cb-7928-403e-a014-843a09be2eda-kube-api-access-24hjs\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:34.476766 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:34.476736    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:34.493418 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:34.493378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:34.496832 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:34.496416    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls\") pod \"maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:34.648972 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:34.648933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:34.804338 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:34.804299    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"]
Jul 15 12:48:34.805351 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:48:34.805325    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fda6cb_7928_403e_a014_843a09be2eda.slice/crio-15b07a5e5057dca941559dfb5cb1be597a5023fc2b36233e62f5eb8e1b73dba3 WatchSource:0}: Error finding container 15b07a5e5057dca941559dfb5cb1be597a5023fc2b36233e62f5eb8e1b73dba3: Status 404 returned error can't find the container with id 15b07a5e5057dca941559dfb5cb1be597a5023fc2b36233e62f5eb8e1b73dba3
Jul 15 12:48:35.512884 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:35.512848    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9" event={"ID":"13fda6cb-7928-403e-a014-843a09be2eda","Type":"ContainerStarted","Data":"21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf"}
Jul 15 12:48:35.513065 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:35.512931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9" event={"ID":"13fda6cb-7928-403e-a014-843a09be2eda","Type":"ContainerStarted","Data":"15b07a5e5057dca941559dfb5cb1be597a5023fc2b36233e62f5eb8e1b73dba3"}
Jul 15 12:48:35.513065 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:35.512956    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:35.534656 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:35.534595    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9" podStartSLOduration=2.184320647 podStartE2EDuration="2.53457416s" totalImagesPullingTime="350.253513ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:48:35.534230475 +0000 UTC m=+2446.634179815" watchObservedRunningTime="2026-07-15 12:48:35.53457416 +0000 UTC m=+2446.634523500"
Jul 15 12:48:41.525744 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:41.525702    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:48:46.281346 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.281310    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"]
Jul 15 12:48:46.285392 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.285369    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"
Jul 15 12:48:46.287782 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.287760    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:48:46.294618 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.294595    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"]
Jul 15 12:48:46.380378 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.380339    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a148ec-ca8c-404e-abd9-b7d890e2794c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx\" (UID: \"57a148ec-ca8c-404e-abd9-b7d890e2794c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"
Jul 15 12:48:46.480986 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.480950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a148ec-ca8c-404e-abd9-b7d890e2794c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx\" (UID: \"57a148ec-ca8c-404e-abd9-b7d890e2794c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"
Jul 15 12:48:46.481627 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.481603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a148ec-ca8c-404e-abd9-b7d890e2794c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx\" (UID: \"57a148ec-ca8c-404e-abd9-b7d890e2794c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"
Jul 15 12:48:46.597792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.597691    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"
Jul 15 12:48:46.736474 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:46.736439    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"]
Jul 15 12:48:46.737463 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:48:46.737441    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a148ec_ca8c_404e_abd9_b7d890e2794c.slice/crio-e55147ad6ed31f3d7bfe25dd674d6d09f20965e4acbec17bb447377a8b4a498f WatchSource:0}: Error finding container e55147ad6ed31f3d7bfe25dd674d6d09f20965e4acbec17bb447377a8b4a498f: Status 404 returned error can't find the container with id e55147ad6ed31f3d7bfe25dd674d6d09f20965e4acbec17bb447377a8b4a498f
Jul 15 12:48:47.585499 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:47.585456    2582 generic.go:356] "Generic (PLEG): container finished" podID="57a148ec-ca8c-404e-abd9-b7d890e2794c" containerID="2676eacba530ed68e77be524667c7a9f90dec199e816579afe64684ea526ef13" exitCode=0
Jul 15 12:48:47.585953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:47.585541    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx" event={"ID":"57a148ec-ca8c-404e-abd9-b7d890e2794c","Type":"ContainerDied","Data":"2676eacba530ed68e77be524667c7a9f90dec199e816579afe64684ea526ef13"}
Jul 15 12:48:47.585953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:47.585597    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx" event={"ID":"57a148ec-ca8c-404e-abd9-b7d890e2794c","Type":"ContainerStarted","Data":"e55147ad6ed31f3d7bfe25dd674d6d09f20965e4acbec17bb447377a8b4a498f"}
Jul 15 12:48:48.726060 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:48.726036    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"
Jul 15 12:48:48.800219 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:48.800185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a148ec-ca8c-404e-abd9-b7d890e2794c-maas-api-service-ca\") pod \"57a148ec-ca8c-404e-abd9-b7d890e2794c\" (UID: \"57a148ec-ca8c-404e-abd9-b7d890e2794c\") "
Jul 15 12:48:48.800550 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:48.800529    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57a148ec-ca8c-404e-abd9-b7d890e2794c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "57a148ec-ca8c-404e-abd9-b7d890e2794c" (UID: "57a148ec-ca8c-404e-abd9-b7d890e2794c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:48:48.901156 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:48.901058    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a148ec-ca8c-404e-abd9-b7d890e2794c-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:48:49.597792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:49.597758    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx"
Jul 15 12:48:49.597792 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:49.597765    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx" event={"ID":"57a148ec-ca8c-404e-abd9-b7d890e2794c","Type":"ContainerDied","Data":"e55147ad6ed31f3d7bfe25dd674d6d09f20965e4acbec17bb447377a8b4a498f"}
Jul 15 12:48:49.598001 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:49.597815    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e55147ad6ed31f3d7bfe25dd674d6d09f20965e4acbec17bb447377a8b4a498f"
Jul 15 12:48:57.162653 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.162618    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"]
Jul 15 12:48:57.163175 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.162938    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455" podUID="f9f13e3f-f538-4611-8cc7-ad0937f86a44" containerName="maas-api" containerID="cri-o://9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557" gracePeriod=30
Jul 15 12:48:57.418193 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.418136    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:57.482340 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.482315    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls\") pod \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") "
Jul 15 12:48:57.482498 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.482373    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f13e3f-f538-4611-8cc7-ad0937f86a44-openshift-service-ca\") pod \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") "
Jul 15 12:48:57.482498 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.482399    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99jd9\" (UniqueName: \"kubernetes.io/projected/f9f13e3f-f538-4611-8cc7-ad0937f86a44-kube-api-access-99jd9\") pod \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\" (UID: \"f9f13e3f-f538-4611-8cc7-ad0937f86a44\") "
Jul 15 12:48:57.482753 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.482726    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f13e3f-f538-4611-8cc7-ad0937f86a44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9f13e3f-f538-4611-8cc7-ad0937f86a44" (UID: "f9f13e3f-f538-4611-8cc7-ad0937f86a44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:48:57.484392 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.484373    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f13e3f-f538-4611-8cc7-ad0937f86a44-kube-api-access-99jd9" (OuterVolumeSpecName: "kube-api-access-99jd9") pod "f9f13e3f-f538-4611-8cc7-ad0937f86a44" (UID: "f9f13e3f-f538-4611-8cc7-ad0937f86a44"). InnerVolumeSpecName "kube-api-access-99jd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:48:57.484499 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.484485    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9f13e3f-f538-4611-8cc7-ad0937f86a44" (UID: "f9f13e3f-f538-4611-8cc7-ad0937f86a44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:48:57.583778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.583746    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9f13e3f-f538-4611-8cc7-ad0937f86a44-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:48:57.583778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.583772    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f13e3f-f538-4611-8cc7-ad0937f86a44-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:48:57.583778 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.583781    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99jd9\" (UniqueName: \"kubernetes.io/projected/f9f13e3f-f538-4611-8cc7-ad0937f86a44-kube-api-access-99jd9\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:48:57.644055 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.644019    2582 generic.go:356] "Generic (PLEG): container finished" podID="f9f13e3f-f538-4611-8cc7-ad0937f86a44" containerID="9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557" exitCode=0
Jul 15 12:48:57.644240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.644062    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"
Jul 15 12:48:57.644240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.644118    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455" event={"ID":"f9f13e3f-f538-4611-8cc7-ad0937f86a44","Type":"ContainerDied","Data":"9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557"}
Jul 15 12:48:57.644240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.644173    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455" event={"ID":"f9f13e3f-f538-4611-8cc7-ad0937f86a44","Type":"ContainerDied","Data":"e289abd7a4a285325171e3fd398f45300b3284fbbf88aa3c2e2373274e2a6c29"}
Jul 15 12:48:57.644240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.644195    2582 scope.go:117] "RemoveContainer" containerID="9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557"
Jul 15 12:48:57.654196 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.654184    2582 scope.go:117] "RemoveContainer" containerID="9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557"
Jul 15 12:48:57.654423 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:48:57.654405    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557\": container with ID starting with 9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557 not found: ID does not exist" containerID="9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557"
Jul 15 12:48:57.654470 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.654429    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557"} err="failed to get container status \"9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557\": rpc error: code = NotFound desc = could not find container \"9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557\": container with ID starting with 9c2e4a1cfd8982eba1949fbd19e062fbf4953581b0388a218916ec8579d82557 not found: ID does not exist"
Jul 15 12:48:57.674402 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.674340    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"]
Jul 15 12:48:57.676847 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:57.676828    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a77f45-58654568d4-7j455"]
Jul 15 12:48:58.062817 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:48:58.062786    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9f13e3f-f538-4611-8cc7-ad0937f86a44" path="/var/lib/kubelet/pods/f9f13e3f-f538-4611-8cc7-ad0937f86a44/volumes"
Jul 15 12:49:12.600738 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.600701    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"]
Jul 15 12:49:12.601209 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.601111    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9f13e3f-f538-4611-8cc7-ad0937f86a44" containerName="maas-api"
Jul 15 12:49:12.601209 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.601124    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9f13e3f-f538-4611-8cc7-ad0937f86a44" containerName="maas-api"
Jul 15 12:49:12.601209 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.601149    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a148ec-ca8c-404e-abd9-b7d890e2794c" containerName="revoke-keys"
Jul 15 12:49:12.601209 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.601156    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a148ec-ca8c-404e-abd9-b7d890e2794c" containerName="revoke-keys"
Jul 15 12:49:12.601340 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.601214    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a148ec-ca8c-404e-abd9-b7d890e2794c" containerName="revoke-keys"
Jul 15 12:49:12.601340 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.601226    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9f13e3f-f538-4611-8cc7-ad0937f86a44" containerName="maas-api"
Jul 15 12:49:12.605765 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.605749    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"
Jul 15 12:49:12.609334 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.609310    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:49:12.619901 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.619875    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"]
Jul 15 12:49:12.722425 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.722382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da4dac31-369d-4a17-9196-c128cb707c45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn\" (UID: \"da4dac31-369d-4a17-9196-c128cb707c45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"
Jul 15 12:49:12.823463 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.823420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da4dac31-369d-4a17-9196-c128cb707c45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn\" (UID: \"da4dac31-369d-4a17-9196-c128cb707c45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"
Jul 15 12:49:12.824032 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.824011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da4dac31-369d-4a17-9196-c128cb707c45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn\" (UID: \"da4dac31-369d-4a17-9196-c128cb707c45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"
Jul 15 12:49:12.915990 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:12.915912    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"
Jul 15 12:49:13.058440 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:13.058418    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"]
Jul 15 12:49:13.060189 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:49:13.060163    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4dac31_369d_4a17_9196_c128cb707c45.slice/crio-00d76696445499807b4d7729ea02ae73133b00ea8edfc8d97e9640dc146564e0 WatchSource:0}: Error finding container 00d76696445499807b4d7729ea02ae73133b00ea8edfc8d97e9640dc146564e0: Status 404 returned error can't find the container with id 00d76696445499807b4d7729ea02ae73133b00ea8edfc8d97e9640dc146564e0
Jul 15 12:49:13.739630 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:13.739584    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn" event={"ID":"da4dac31-369d-4a17-9196-c128cb707c45","Type":"ContainerStarted","Data":"6389a4e196034f147435a8fb9beb3114610056f4938f3eb65c41bcdef9ba51ef"}
Jul 15 12:49:13.740141 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:13.739662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn" event={"ID":"da4dac31-369d-4a17-9196-c128cb707c45","Type":"ContainerStarted","Data":"00d76696445499807b4d7729ea02ae73133b00ea8edfc8d97e9640dc146564e0"}
Jul 15 12:49:13.767020 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:13.766971    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn" podStartSLOduration=1.766956302 podStartE2EDuration="1.766956302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:49:13.764801138 +0000 UTC m=+2484.864750488" watchObservedRunningTime="2026-07-15 12:49:13.766956302 +0000 UTC m=+2484.866905682"
Jul 15 12:49:14.747191 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:14.747158    2582 generic.go:356] "Generic (PLEG): container finished" podID="da4dac31-369d-4a17-9196-c128cb707c45" containerID="6389a4e196034f147435a8fb9beb3114610056f4938f3eb65c41bcdef9ba51ef" exitCode=0
Jul 15 12:49:14.747601 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:14.747246    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn" event={"ID":"da4dac31-369d-4a17-9196-c128cb707c45","Type":"ContainerDied","Data":"6389a4e196034f147435a8fb9beb3114610056f4938f3eb65c41bcdef9ba51ef"}
Jul 15 12:49:15.893701 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:15.893677    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"
Jul 15 12:49:16.051743 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:16.051659    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da4dac31-369d-4a17-9196-c128cb707c45-maas-api-service-ca\") pod \"da4dac31-369d-4a17-9196-c128cb707c45\" (UID: \"da4dac31-369d-4a17-9196-c128cb707c45\") "
Jul 15 12:49:16.051994 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:16.051974    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da4dac31-369d-4a17-9196-c128cb707c45-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da4dac31-369d-4a17-9196-c128cb707c45" (UID: "da4dac31-369d-4a17-9196-c128cb707c45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:49:16.152322 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:16.152286    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da4dac31-369d-4a17-9196-c128cb707c45-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:49:16.761108 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:16.761073    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn"
Jul 15 12:49:16.761279 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:16.761075    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn" event={"ID":"da4dac31-369d-4a17-9196-c128cb707c45","Type":"ContainerDied","Data":"00d76696445499807b4d7729ea02ae73133b00ea8edfc8d97e9640dc146564e0"}
Jul 15 12:49:16.761279 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:16.761217    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00d76696445499807b4d7729ea02ae73133b00ea8edfc8d97e9640dc146564e0"
Jul 15 12:49:23.199423 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.199389    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"]
Jul 15 12:49:23.199873 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.199644    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9" podUID="13fda6cb-7928-403e-a014-843a09be2eda" containerName="maas-api" containerID="cri-o://21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf" gracePeriod=30
Jul 15 12:49:23.456629 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.456605    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:49:23.624764 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.624738    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24hjs\" (UniqueName: \"kubernetes.io/projected/13fda6cb-7928-403e-a014-843a09be2eda-kube-api-access-24hjs\") pod \"13fda6cb-7928-403e-a014-843a09be2eda\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") "
Jul 15 12:49:23.624764 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.624768    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fda6cb-7928-403e-a014-843a09be2eda-openshift-service-ca\") pod \"13fda6cb-7928-403e-a014-843a09be2eda\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") "
Jul 15 12:49:23.624962 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.624810    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls\") pod \"13fda6cb-7928-403e-a014-843a09be2eda\" (UID: \"13fda6cb-7928-403e-a014-843a09be2eda\") "
Jul 15 12:49:23.625171 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.625136    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13fda6cb-7928-403e-a014-843a09be2eda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13fda6cb-7928-403e-a014-843a09be2eda" (UID: "13fda6cb-7928-403e-a014-843a09be2eda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:49:23.626823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.626799    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13fda6cb-7928-403e-a014-843a09be2eda-kube-api-access-24hjs" (OuterVolumeSpecName: "kube-api-access-24hjs") pod "13fda6cb-7928-403e-a014-843a09be2eda" (UID: "13fda6cb-7928-403e-a014-843a09be2eda"). InnerVolumeSpecName "kube-api-access-24hjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:49:23.626937 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.626858    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13fda6cb-7928-403e-a014-843a09be2eda" (UID: "13fda6cb-7928-403e-a014-843a09be2eda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:49:23.725794 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.725729    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fda6cb-7928-403e-a014-843a09be2eda-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:49:23.725794 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.725755    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24hjs\" (UniqueName: \"kubernetes.io/projected/13fda6cb-7928-403e-a014-843a09be2eda-kube-api-access-24hjs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:49:23.725794 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.725765    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fda6cb-7928-403e-a014-843a09be2eda-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:49:23.802775 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.802744    2582 generic.go:356] "Generic (PLEG): container finished" podID="13fda6cb-7928-403e-a014-843a09be2eda" containerID="21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf" exitCode=0
Jul 15 12:49:23.802923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.802791    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"
Jul 15 12:49:23.802923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.802834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9" event={"ID":"13fda6cb-7928-403e-a014-843a09be2eda","Type":"ContainerDied","Data":"21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf"}
Jul 15 12:49:23.802923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.802888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9" event={"ID":"13fda6cb-7928-403e-a014-843a09be2eda","Type":"ContainerDied","Data":"15b07a5e5057dca941559dfb5cb1be597a5023fc2b36233e62f5eb8e1b73dba3"}
Jul 15 12:49:23.802923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.802917    2582 scope.go:117] "RemoveContainer" containerID="21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf"
Jul 15 12:49:23.815033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.815012    2582 scope.go:117] "RemoveContainer" containerID="21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf"
Jul 15 12:49:23.815482 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:49:23.815461    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf\": container with ID starting with 21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf not found: ID does not exist" containerID="21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf"
Jul 15 12:49:23.815581 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.815486    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf"} err="failed to get container status \"21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf\": rpc error: code = NotFound desc = could not find container \"21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf\": container with ID starting with 21f4eb5b12db36847d8c3674abdf161e69ea996f5687616df2af15549d3902cf not found: ID does not exist"
Jul 15 12:49:23.829063 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.829038    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"]
Jul 15 12:49:23.833763 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:23.833743    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a2a9e1-5964ff64f4-mrbm9"]
Jul 15 12:49:24.063615 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:24.063536    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13fda6cb-7928-403e-a014-843a09be2eda" path="/var/lib/kubelet/pods/13fda6cb-7928-403e-a014-843a09be2eda/volumes"
Jul 15 12:49:43.105417 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.105338    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"]
Jul 15 12:49:43.105942 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.105925    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13fda6cb-7928-403e-a014-843a09be2eda" containerName="maas-api"
Jul 15 12:49:43.105997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.105945    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13fda6cb-7928-403e-a014-843a09be2eda" containerName="maas-api"
Jul 15 12:49:43.105997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.105984    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da4dac31-369d-4a17-9196-c128cb707c45" containerName="revoke-keys"
Jul 15 12:49:43.105997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.105993    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4dac31-369d-4a17-9196-c128cb707c45" containerName="revoke-keys"
Jul 15 12:49:43.106109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.106083    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="da4dac31-369d-4a17-9196-c128cb707c45" containerName="revoke-keys"
Jul 15 12:49:43.106152 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.106114    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13fda6cb-7928-403e-a014-843a09be2eda" containerName="maas-api"
Jul 15 12:49:43.109967 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.109941    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.112315 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.112287    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-abb283-serving-cert\""
Jul 15 12:49:43.112434 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.112423    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbtbf\""
Jul 15 12:49:43.122663 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.122632    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"]
Jul 15 12:49:43.181549 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.181516    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b598aa-273c-4899-88d2-906f9ce3f333-openshift-service-ca\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.181708 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.181553    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.181708 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.181634    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfdbw\" (UniqueName: \"kubernetes.io/projected/e1b598aa-273c-4899-88d2-906f9ce3f333-kube-api-access-jfdbw\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.282516 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.282481    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfdbw\" (UniqueName: \"kubernetes.io/projected/e1b598aa-273c-4899-88d2-906f9ce3f333-kube-api-access-jfdbw\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.282690 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.282639    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b598aa-273c-4899-88d2-906f9ce3f333-openshift-service-ca\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.282690 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.282670    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.282864 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:49:43.282848    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-serving-cert: secret "maas-api-e2e-shared-a-abb283-serving-cert" not found
Jul 15 12:49:43.282934 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:49:43.282924    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls podName:e1b598aa-273c-4899-88d2-906f9ce3f333 nodeName:}" failed. No retries permitted until 2026-07-15 12:49:43.782905597 +0000 UTC m=+2514.882854915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls") pod "maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" (UID: "e1b598aa-273c-4899-88d2-906f9ce3f333") : secret "maas-api-e2e-shared-a-abb283-serving-cert" not found
Jul 15 12:49:43.283291 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.283271    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b598aa-273c-4899-88d2-906f9ce3f333-openshift-service-ca\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.292382 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.292356    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfdbw\" (UniqueName: \"kubernetes.io/projected/e1b598aa-273c-4899-88d2-906f9ce3f333-kube-api-access-jfdbw\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.786717 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.786680    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:43.789035 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:43.788996    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls\") pod \"maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:44.020763 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:44.020728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:44.173323 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:44.173294    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"]
Jul 15 12:49:44.174141 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:49:44.174085    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b598aa_273c_4899_88d2_906f9ce3f333.slice/crio-1f94f6875b4b956f92f4364bc42e846042074768c16665159e874fcebd9d623c WatchSource:0}: Error finding container 1f94f6875b4b956f92f4364bc42e846042074768c16665159e874fcebd9d623c: Status 404 returned error can't find the container with id 1f94f6875b4b956f92f4364bc42e846042074768c16665159e874fcebd9d623c
Jul 15 12:49:44.939305 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:44.939259    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" event={"ID":"e1b598aa-273c-4899-88d2-906f9ce3f333","Type":"ContainerStarted","Data":"c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda"}
Jul 15 12:49:44.939487 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:44.939330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" event={"ID":"e1b598aa-273c-4899-88d2-906f9ce3f333","Type":"ContainerStarted","Data":"1f94f6875b4b956f92f4364bc42e846042074768c16665159e874fcebd9d623c"}
Jul 15 12:49:44.939584 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:44.939547    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:50.953672 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:50.953641    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:49:50.977649 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:50.977563    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" podStartSLOduration=7.438234311 podStartE2EDuration="7.977542175s" totalImagesPullingTime="539.307864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:49:44.976071923 +0000 UTC m=+2516.076021260" watchObservedRunningTime="2026-07-15 12:49:50.977542175 +0000 UTC m=+2522.077491514"
Jul 15 12:49:50.979702 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:50.979676    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"]
Jul 15 12:49:50.985149 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:50.985084    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:50.987376 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:50.987355    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bdc1bb-serving-cert\""
Jul 15 12:49:50.992933 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:50.992913    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"]
Jul 15 12:49:51.051445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.051412    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dcb796-47e5-409a-bc6a-c7308783a558-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.051445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.051442    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ff2c\" (UniqueName: \"kubernetes.io/projected/d5dcb796-47e5-409a-bc6a-c7308783a558-kube-api-access-8ff2c\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.051637 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.051484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.152214 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.152178    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dcb796-47e5-409a-bc6a-c7308783a558-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.152214 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.152216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ff2c\" (UniqueName: \"kubernetes.io/projected/d5dcb796-47e5-409a-bc6a-c7308783a558-kube-api-access-8ff2c\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.152417 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.152271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.152417 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:49:51.152372    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-serving-cert: secret "maas-api-e2e-shared-b-bdc1bb-serving-cert" not found
Jul 15 12:49:51.152484 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:49:51.152440    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls podName:d5dcb796-47e5-409a-bc6a-c7308783a558 nodeName:}" failed. No retries permitted until 2026-07-15 12:49:51.652419084 +0000 UTC m=+2522.752368404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls") pod "maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792" (UID: "d5dcb796-47e5-409a-bc6a-c7308783a558") : secret "maas-api-e2e-shared-b-bdc1bb-serving-cert" not found
Jul 15 12:49:51.152891 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.152872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dcb796-47e5-409a-bc6a-c7308783a558-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.160670 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.160640    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ff2c\" (UniqueName: \"kubernetes.io/projected/d5dcb796-47e5-409a-bc6a-c7308783a558-kube-api-access-8ff2c\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.658297 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.658258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.660573 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.660550    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls\") pod \"maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:51.898780 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:51.898750    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:52.248182 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:52.248157    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"]
Jul 15 12:49:52.250674 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:49:52.250646    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5dcb796_47e5_409a_bc6a_c7308783a558.slice/crio-9b6efc7bd6af6efb185cbd143731cbe6bb62a3aaaf542cd40beb483a708b4313 WatchSource:0}: Error finding container 9b6efc7bd6af6efb185cbd143731cbe6bb62a3aaaf542cd40beb483a708b4313: Status 404 returned error can't find the container with id 9b6efc7bd6af6efb185cbd143731cbe6bb62a3aaaf542cd40beb483a708b4313
Jul 15 12:49:52.991856 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:52.991767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792" event={"ID":"d5dcb796-47e5-409a-bc6a-c7308783a558","Type":"ContainerStarted","Data":"7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e"}
Jul 15 12:49:52.991856 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:52.991818    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792" event={"ID":"d5dcb796-47e5-409a-bc6a-c7308783a558","Type":"ContainerStarted","Data":"9b6efc7bd6af6efb185cbd143731cbe6bb62a3aaaf542cd40beb483a708b4313"}
Jul 15 12:49:52.992063 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:52.991899    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:49:53.010771 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:53.010722    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792" podStartSLOduration=2.531653575 podStartE2EDuration="3.010704619s" totalImagesPullingTime="479.051044ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:49:53.008052232 +0000 UTC m=+2524.108001569" watchObservedRunningTime="2026-07-15 12:49:53.010704619 +0000 UTC m=+2524.110653991"
Jul 15 12:49:59.003536 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:49:59.003507    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:52:49.641912 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:52:49.641876    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:52:49.650945 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:52:49.650923    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:55:51.276204 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.276172    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140062088720048"]
Jul 15 12:55:51.279809 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.279793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:51.282243 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.282217    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mjxfr\""
Jul 15 12:55:51.285500 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.285477    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088720048"]
Jul 15 12:55:51.370517 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.370484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm6xp\" (UniqueName: \"kubernetes.io/projected/27a43ac2-f99f-41fc-be56-e96321be34b2-kube-api-access-vm6xp\") pod \"test-curl-7351-140062088720048\" (UID: \"27a43ac2-f99f-41fc-be56-e96321be34b2\") " pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:51.471404 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.471370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm6xp\" (UniqueName: \"kubernetes.io/projected/27a43ac2-f99f-41fc-be56-e96321be34b2-kube-api-access-vm6xp\") pod \"test-curl-7351-140062088720048\" (UID: \"27a43ac2-f99f-41fc-be56-e96321be34b2\") " pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:51.479827 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.479798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm6xp\" (UniqueName: \"kubernetes.io/projected/27a43ac2-f99f-41fc-be56-e96321be34b2-kube-api-access-vm6xp\") pod \"test-curl-7351-140062088720048\" (UID: \"27a43ac2-f99f-41fc-be56-e96321be34b2\") " pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:51.589913 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.589831    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:51.727736 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.727709    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088720048"]
Jul 15 12:55:51.728865 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:55:51.728833    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27a43ac2_f99f_41fc_be56_e96321be34b2.slice/crio-9239fad0fbb8f4d134e9cba9b6478771545a9f00878ab86796a4b1da39bcc179 WatchSource:0}: Error finding container 9239fad0fbb8f4d134e9cba9b6478771545a9f00878ab86796a4b1da39bcc179: Status 404 returned error can't find the container with id 9239fad0fbb8f4d134e9cba9b6478771545a9f00878ab86796a4b1da39bcc179
Jul 15 12:55:51.730310 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:51.730294    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:55:52.159160 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:52.159126    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088720048" event={"ID":"27a43ac2-f99f-41fc-be56-e96321be34b2","Type":"ContainerStarted","Data":"9239fad0fbb8f4d134e9cba9b6478771545a9f00878ab86796a4b1da39bcc179"}
Jul 15 12:55:53.166407 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:53.166372    2582 generic.go:356] "Generic (PLEG): container finished" podID="27a43ac2-f99f-41fc-be56-e96321be34b2" containerID="133af1fca2c9f92b1dfbe04588952654918d4362ea20351a8b8df159bf0320d2" exitCode=0
Jul 15 12:55:53.166825 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:53.166458    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088720048" event={"ID":"27a43ac2-f99f-41fc-be56-e96321be34b2","Type":"ContainerDied","Data":"133af1fca2c9f92b1dfbe04588952654918d4362ea20351a8b8df159bf0320d2"}
Jul 15 12:55:54.314307 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.314278    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:54.362162 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.362133    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140062088720048_27a43ac2-f99f-41fc-be56-e96321be34b2/test-curl-7351-140062088720048/0.log"
Jul 15 12:55:54.389380 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.389342    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140062088720048"]
Jul 15 12:55:54.393702 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.393679    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140062088720048"]
Jul 15 12:55:54.398927 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.398904    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm6xp\" (UniqueName: \"kubernetes.io/projected/27a43ac2-f99f-41fc-be56-e96321be34b2-kube-api-access-vm6xp\") pod \"27a43ac2-f99f-41fc-be56-e96321be34b2\" (UID: \"27a43ac2-f99f-41fc-be56-e96321be34b2\") "
Jul 15 12:55:54.400939 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.400912    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a43ac2-f99f-41fc-be56-e96321be34b2-kube-api-access-vm6xp" (OuterVolumeSpecName: "kube-api-access-vm6xp") pod "27a43ac2-f99f-41fc-be56-e96321be34b2" (UID: "27a43ac2-f99f-41fc-be56-e96321be34b2"). InnerVolumeSpecName "kube-api-access-vm6xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:55:54.500434 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.500403    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm6xp\" (UniqueName: \"kubernetes.io/projected/27a43ac2-f99f-41fc-be56-e96321be34b2-kube-api-access-vm6xp\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:55:54.521584 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.521552    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140062088718896"]
Jul 15 12:55:54.521985 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.521970    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a43ac2-f99f-41fc-be56-e96321be34b2" containerName="test-curl-7351-140062088720048"
Jul 15 12:55:54.522040 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.521987    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a43ac2-f99f-41fc-be56-e96321be34b2" containerName="test-curl-7351-140062088720048"
Jul 15 12:55:54.522106 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.522081    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="27a43ac2-f99f-41fc-be56-e96321be34b2" containerName="test-curl-7351-140062088720048"
Jul 15 12:55:54.525238 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.525218    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088718896"
Jul 15 12:55:54.533319 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.533296    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088718896"]
Jul 15 12:55:54.601687 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.601648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht7sj\" (UniqueName: \"kubernetes.io/projected/3818944d-f4ae-40d3-bea2-32f9c63f9a79-kube-api-access-ht7sj\") pod \"test-curl-7351-140062088718896\" (UID: \"3818944d-f4ae-40d3-bea2-32f9c63f9a79\") " pod="opendatahub/test-curl-7351-140062088718896"
Jul 15 12:55:54.702630 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.702590    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht7sj\" (UniqueName: \"kubernetes.io/projected/3818944d-f4ae-40d3-bea2-32f9c63f9a79-kube-api-access-ht7sj\") pod \"test-curl-7351-140062088718896\" (UID: \"3818944d-f4ae-40d3-bea2-32f9c63f9a79\") " pod="opendatahub/test-curl-7351-140062088718896"
Jul 15 12:55:54.712196 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.712170    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht7sj\" (UniqueName: \"kubernetes.io/projected/3818944d-f4ae-40d3-bea2-32f9c63f9a79-kube-api-access-ht7sj\") pod \"test-curl-7351-140062088718896\" (UID: \"3818944d-f4ae-40d3-bea2-32f9c63f9a79\") " pod="opendatahub/test-curl-7351-140062088718896"
Jul 15 12:55:54.835864 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.835773    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088718896"
Jul 15 12:55:54.984282 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:54.984255    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088718896"]
Jul 15 12:55:54.985652 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:55:54.985624    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3818944d_f4ae_40d3_bea2_32f9c63f9a79.slice/crio-03adb4f3f5dcbb664b57f0166ad8f1b395de492132ef4dc4c4bd2e051e492cca WatchSource:0}: Error finding container 03adb4f3f5dcbb664b57f0166ad8f1b395de492132ef4dc4c4bd2e051e492cca: Status 404 returned error can't find the container with id 03adb4f3f5dcbb664b57f0166ad8f1b395de492132ef4dc4c4bd2e051e492cca
Jul 15 12:55:55.180652 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:55.180571    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:55.180652 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:55.180572    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9239fad0fbb8f4d134e9cba9b6478771545a9f00878ab86796a4b1da39bcc179"
Jul 15 12:55:55.181896 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:55.181868    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088718896" event={"ID":"3818944d-f4ae-40d3-bea2-32f9c63f9a79","Type":"ContainerStarted","Data":"03adb4f3f5dcbb664b57f0166ad8f1b395de492132ef4dc4c4bd2e051e492cca"}
Jul 15 12:55:55.194581 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:55:55.194552    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7351-140062088720048\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="27a43ac2-f99f-41fc-be56-e96321be34b2" pod="opendatahub/test-curl-7351-140062088720048"
Jul 15 12:55:56.062911 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:56.062881    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27a43ac2-f99f-41fc-be56-e96321be34b2" path="/var/lib/kubelet/pods/27a43ac2-f99f-41fc-be56-e96321be34b2/volumes"
Jul 15 12:55:56.189596 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:56.189553    2582 generic.go:356] "Generic (PLEG): container finished" podID="3818944d-f4ae-40d3-bea2-32f9c63f9a79" containerID="3dd0052c1cbab10c5d42c542a38bdd7ce9b7bf7a447a26eb6a5c3fd5244cc7c9" exitCode=0
Jul 15 12:55:56.189755 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:56.189637    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088718896" event={"ID":"3818944d-f4ae-40d3-bea2-32f9c63f9a79","Type":"ContainerDied","Data":"3dd0052c1cbab10c5d42c542a38bdd7ce9b7bf7a447a26eb6a5c3fd5244cc7c9"}
Jul 15 12:55:57.349755 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.349731    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088718896"
Jul 15 12:55:57.372059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.372024    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140062088718896_3818944d-f4ae-40d3-bea2-32f9c63f9a79/test-curl-7351-140062088718896/0.log"
Jul 15 12:55:57.397287 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.397244    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140062088718896"]
Jul 15 12:55:57.403817 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.403787    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140062088718896"]
Jul 15 12:55:57.427997 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.427965    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht7sj\" (UniqueName: \"kubernetes.io/projected/3818944d-f4ae-40d3-bea2-32f9c63f9a79-kube-api-access-ht7sj\") pod \"3818944d-f4ae-40d3-bea2-32f9c63f9a79\" (UID: \"3818944d-f4ae-40d3-bea2-32f9c63f9a79\") "
Jul 15 12:55:57.430513 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.430481    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3818944d-f4ae-40d3-bea2-32f9c63f9a79-kube-api-access-ht7sj" (OuterVolumeSpecName: "kube-api-access-ht7sj") pod "3818944d-f4ae-40d3-bea2-32f9c63f9a79" (UID: "3818944d-f4ae-40d3-bea2-32f9c63f9a79"). InnerVolumeSpecName "kube-api-access-ht7sj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:55:57.528838 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.528799    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht7sj\" (UniqueName: \"kubernetes.io/projected/3818944d-f4ae-40d3-bea2-32f9c63f9a79-kube-api-access-ht7sj\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:55:57.577919 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.577882    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:55:57.578325 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.578311    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3818944d-f4ae-40d3-bea2-32f9c63f9a79" containerName="test-curl-7351-140062088718896"
Jul 15 12:55:57.578376 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.578326    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3818944d-f4ae-40d3-bea2-32f9c63f9a79" containerName="test-curl-7351-140062088718896"
Jul 15 12:55:57.578412 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.578402    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3818944d-f4ae-40d3-bea2-32f9c63f9a79" containerName="test-curl-7351-140062088718896"
Jul 15 12:55:57.581612 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.581597    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:55:57.587483 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.587460    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:55:57.629670 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.629634    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktwrw\" (UniqueName: \"kubernetes.io/projected/2c46b4e6-d3dd-4a3a-95b0-062dbae14da3-kube-api-access-ktwrw\") pod \"test-curl-7351-140062088578800\" (UID: \"2c46b4e6-d3dd-4a3a-95b0-062dbae14da3\") " pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:55:57.730675 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.730633    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktwrw\" (UniqueName: \"kubernetes.io/projected/2c46b4e6-d3dd-4a3a-95b0-062dbae14da3-kube-api-access-ktwrw\") pod \"test-curl-7351-140062088578800\" (UID: \"2c46b4e6-d3dd-4a3a-95b0-062dbae14da3\") " pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:55:57.739489 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.739453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktwrw\" (UniqueName: \"kubernetes.io/projected/2c46b4e6-d3dd-4a3a-95b0-062dbae14da3-kube-api-access-ktwrw\") pod \"test-curl-7351-140062088578800\" (UID: \"2c46b4e6-d3dd-4a3a-95b0-062dbae14da3\") " pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:55:57.892711 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:57.892623    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:55:58.034841 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:58.034813    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:55:58.036754 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:55:58.036726    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c46b4e6_d3dd_4a3a_95b0_062dbae14da3.slice/crio-9ab589eb80461cc5787358cd12fd1012871afb46f8e963e60011aaf20e2ef470 WatchSource:0}: Error finding container 9ab589eb80461cc5787358cd12fd1012871afb46f8e963e60011aaf20e2ef470: Status 404 returned error can't find the container with id 9ab589eb80461cc5787358cd12fd1012871afb46f8e963e60011aaf20e2ef470
Jul 15 12:55:58.063616 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:58.063586    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3818944d-f4ae-40d3-bea2-32f9c63f9a79" path="/var/lib/kubelet/pods/3818944d-f4ae-40d3-bea2-32f9c63f9a79/volumes"
Jul 15 12:55:58.202388 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:58.202353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088578800" event={"ID":"2c46b4e6-d3dd-4a3a-95b0-062dbae14da3","Type":"ContainerStarted","Data":"9ab589eb80461cc5787358cd12fd1012871afb46f8e963e60011aaf20e2ef470"}
Jul 15 12:55:58.203801 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:58.203782    2582 scope.go:117] "RemoveContainer" containerID="3dd0052c1cbab10c5d42c542a38bdd7ce9b7bf7a447a26eb6a5c3fd5244cc7c9"
Jul 15 12:55:58.203801 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:58.203792    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088718896"
Jul 15 12:55:59.212941 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:59.212909    2582 generic.go:356] "Generic (PLEG): container finished" podID="2c46b4e6-d3dd-4a3a-95b0-062dbae14da3" containerID="7f765d6736eea32334309fe4a13d05f028cd0599ebdb6f5f8f94b0cc46bbf3a7" exitCode=0
Jul 15 12:55:59.213338 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:55:59.212978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088578800" event={"ID":"2c46b4e6-d3dd-4a3a-95b0-062dbae14da3","Type":"ContainerDied","Data":"7f765d6736eea32334309fe4a13d05f028cd0599ebdb6f5f8f94b0cc46bbf3a7"}
Jul 15 12:56:00.367033 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.367000    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:00.389849 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.389816    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140062088578800_2c46b4e6-d3dd-4a3a-95b0-062dbae14da3/test-curl-7351-140062088578800/0.log"
Jul 15 12:56:00.414696 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.414664    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:56:00.419982 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.419955    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:56:00.458777 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.458746    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktwrw\" (UniqueName: \"kubernetes.io/projected/2c46b4e6-d3dd-4a3a-95b0-062dbae14da3-kube-api-access-ktwrw\") pod \"2c46b4e6-d3dd-4a3a-95b0-062dbae14da3\" (UID: \"2c46b4e6-d3dd-4a3a-95b0-062dbae14da3\") "
Jul 15 12:56:00.460867 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.460838    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c46b4e6-d3dd-4a3a-95b0-062dbae14da3-kube-api-access-ktwrw" (OuterVolumeSpecName: "kube-api-access-ktwrw") pod "2c46b4e6-d3dd-4a3a-95b0-062dbae14da3" (UID: "2c46b4e6-d3dd-4a3a-95b0-062dbae14da3"). InnerVolumeSpecName "kube-api-access-ktwrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:56:00.540386 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.540295    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:56:00.540774 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.540757    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c46b4e6-d3dd-4a3a-95b0-062dbae14da3" containerName="test-curl-7351-140062088578800"
Jul 15 12:56:00.540774 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.540774    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c46b4e6-d3dd-4a3a-95b0-062dbae14da3" containerName="test-curl-7351-140062088578800"
Jul 15 12:56:00.540860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.540841    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c46b4e6-d3dd-4a3a-95b0-062dbae14da3" containerName="test-curl-7351-140062088578800"
Jul 15 12:56:00.544055 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.544034    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:00.552644 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.552614    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:56:00.559580 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.559547    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktwrw\" (UniqueName: \"kubernetes.io/projected/2c46b4e6-d3dd-4a3a-95b0-062dbae14da3-kube-api-access-ktwrw\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:00.660290 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.660256    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv27b\" (UniqueName: \"kubernetes.io/projected/c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c-kube-api-access-wv27b\") pod \"test-curl-7351-140062088578800\" (UID: \"c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c\") " pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:00.761172 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.761131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv27b\" (UniqueName: \"kubernetes.io/projected/c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c-kube-api-access-wv27b\") pod \"test-curl-7351-140062088578800\" (UID: \"c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c\") " pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:00.769699 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.769663    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv27b\" (UniqueName: \"kubernetes.io/projected/c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c-kube-api-access-wv27b\") pod \"test-curl-7351-140062088578800\" (UID: \"c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c\") " pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:00.855277 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.855187    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:00.998575 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:00.998548    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:56:00.999716 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:56:00.999689    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5424cf5_c5ad_42c1_9cdf_ddeec462bc9c.slice/crio-02f7362a0f8f4c82cd8d14f106985f37a2e0175721556145f375305481221bb9 WatchSource:0}: Error finding container 02f7362a0f8f4c82cd8d14f106985f37a2e0175721556145f375305481221bb9: Status 404 returned error can't find the container with id 02f7362a0f8f4c82cd8d14f106985f37a2e0175721556145f375305481221bb9
Jul 15 12:56:01.235123 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:01.235066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088578800" event={"ID":"c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c","Type":"ContainerStarted","Data":"02f7362a0f8f4c82cd8d14f106985f37a2e0175721556145f375305481221bb9"}
Jul 15 12:56:01.236428 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:01.236403    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ab589eb80461cc5787358cd12fd1012871afb46f8e963e60011aaf20e2ef470"
Jul 15 12:56:01.236428 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:01.236414    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:01.248411 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:01.248377    2582 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7351-140062088578800" oldPodUID="2c46b4e6-d3dd-4a3a-95b0-062dbae14da3" podUID="c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c"
Jul 15 12:56:02.063432 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:02.063400    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c46b4e6-d3dd-4a3a-95b0-062dbae14da3" path="/var/lib/kubelet/pods/2c46b4e6-d3dd-4a3a-95b0-062dbae14da3/volumes"
Jul 15 12:56:02.244379 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:02.244343    2582 generic.go:356] "Generic (PLEG): container finished" podID="c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c" containerID="2dedd7651972840c51ed592f39d1bc022d3a5fcd40b8b83f18ab726386cdc554" exitCode=0
Jul 15 12:56:02.244548 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:02.244430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062088578800" event={"ID":"c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c","Type":"ContainerDied","Data":"2dedd7651972840c51ed592f39d1bc022d3a5fcd40b8b83f18ab726386cdc554"}
Jul 15 12:56:03.394899 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.394872    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:03.416796 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.416767    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140062088578800_c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c/test-curl-7351-140062088578800/0.log"
Jul 15 12:56:03.447037 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.447001    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:56:03.450038 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.450014    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140062088578800"]
Jul 15 12:56:03.487603 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.487566    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv27b\" (UniqueName: \"kubernetes.io/projected/c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c-kube-api-access-wv27b\") pod \"c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c\" (UID: \"c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c\") "
Jul 15 12:56:03.489608 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.489580    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c-kube-api-access-wv27b" (OuterVolumeSpecName: "kube-api-access-wv27b") pod "c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c" (UID: "c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c"). InnerVolumeSpecName "kube-api-access-wv27b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:56:03.563526 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.563439    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140062090032448"]
Jul 15 12:56:03.563859 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.563846    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c" containerName="test-curl-7351-140062088578800"
Jul 15 12:56:03.563909 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.563860    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c" containerName="test-curl-7351-140062088578800"
Jul 15 12:56:03.563953 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.563931    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c" containerName="test-curl-7351-140062088578800"
Jul 15 12:56:03.566976 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.566956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:03.576148 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.576121    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062090032448"]
Jul 15 12:56:03.589106 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.589066    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv27b\" (UniqueName: \"kubernetes.io/projected/c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c-kube-api-access-wv27b\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:03.689834 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.689782    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65klv\" (UniqueName: \"kubernetes.io/projected/c5467457-b130-4a87-9c92-f97e210612d4-kube-api-access-65klv\") pod \"test-curl-7351-140062090032448\" (UID: \"c5467457-b130-4a87-9c92-f97e210612d4\") " pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:03.791108 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.791067    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65klv\" (UniqueName: \"kubernetes.io/projected/c5467457-b130-4a87-9c92-f97e210612d4-kube-api-access-65klv\") pod \"test-curl-7351-140062090032448\" (UID: \"c5467457-b130-4a87-9c92-f97e210612d4\") " pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:03.799125 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.799082    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65klv\" (UniqueName: \"kubernetes.io/projected/c5467457-b130-4a87-9c92-f97e210612d4-kube-api-access-65klv\") pod \"test-curl-7351-140062090032448\" (UID: \"c5467457-b130-4a87-9c92-f97e210612d4\") " pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:03.877432 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:03.877340    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:04.023742 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:04.023715    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140062090032448"]
Jul 15 12:56:04.025057 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:56:04.025030    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5467457_b130_4a87_9c92_f97e210612d4.slice/crio-812311e0c4b07ffe7a99d62522f7895e25cf64fa6f88975f99bc2507b9389032 WatchSource:0}: Error finding container 812311e0c4b07ffe7a99d62522f7895e25cf64fa6f88975f99bc2507b9389032: Status 404 returned error can't find the container with id 812311e0c4b07ffe7a99d62522f7895e25cf64fa6f88975f99bc2507b9389032
Jul 15 12:56:04.064079 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:04.064045    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c" path="/var/lib/kubelet/pods/c5424cf5-c5ad-42c1-9cdf-ddeec462bc9c/volumes"
Jul 15 12:56:04.258788 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:04.258749    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062090032448" event={"ID":"c5467457-b130-4a87-9c92-f97e210612d4","Type":"ContainerStarted","Data":"812311e0c4b07ffe7a99d62522f7895e25cf64fa6f88975f99bc2507b9389032"}
Jul 15 12:56:04.259927 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:04.259907    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062088578800"
Jul 15 12:56:04.260058 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:04.259969    2582 scope.go:117] "RemoveContainer" containerID="2dedd7651972840c51ed592f39d1bc022d3a5fcd40b8b83f18ab726386cdc554"
Jul 15 12:56:05.268308 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:05.268274    2582 generic.go:356] "Generic (PLEG): container finished" podID="c5467457-b130-4a87-9c92-f97e210612d4" containerID="525d729422f1b615233eb9a88d1f6c56f7f5176a42453e2ca6dca5b20df9ea6d" exitCode=0
Jul 15 12:56:05.268679 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:05.268314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140062090032448" event={"ID":"c5467457-b130-4a87-9c92-f97e210612d4","Type":"ContainerDied","Data":"525d729422f1b615233eb9a88d1f6c56f7f5176a42453e2ca6dca5b20df9ea6d"}
Jul 15 12:56:06.418738 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:06.418712    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:06.440624 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:06.440585    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140062090032448_c5467457-b130-4a87-9c92-f97e210612d4/test-curl-7351-140062090032448/0.log"
Jul 15 12:56:06.464201 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:06.464164    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140062090032448"]
Jul 15 12:56:06.466967 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:06.466945    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140062090032448"]
Jul 15 12:56:06.516316 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:06.516279    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65klv\" (UniqueName: \"kubernetes.io/projected/c5467457-b130-4a87-9c92-f97e210612d4-kube-api-access-65klv\") pod \"c5467457-b130-4a87-9c92-f97e210612d4\" (UID: \"c5467457-b130-4a87-9c92-f97e210612d4\") "
Jul 15 12:56:06.518407 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:06.518379    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5467457-b130-4a87-9c92-f97e210612d4-kube-api-access-65klv" (OuterVolumeSpecName: "kube-api-access-65klv") pod "c5467457-b130-4a87-9c92-f97e210612d4" (UID: "c5467457-b130-4a87-9c92-f97e210612d4"). InnerVolumeSpecName "kube-api-access-65klv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:56:06.617995 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:06.617907    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65klv\" (UniqueName: \"kubernetes.io/projected/c5467457-b130-4a87-9c92-f97e210612d4-kube-api-access-65klv\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:07.282485 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:07.282458    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:07.282666 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:07.282521    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="812311e0c4b07ffe7a99d62522f7895e25cf64fa6f88975f99bc2507b9389032"
Jul 15 12:56:07.292246 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:56:07.292213    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7351-140062090032448\" is forbidden: User \"system:node:ip-10-0-135-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-186.ec2.internal' and this object" podUID="c5467457-b130-4a87-9c92-f97e210612d4" pod="opendatahub/test-curl-7351-140062090032448"
Jul 15 12:56:08.066153 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:08.066113    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5467457-b130-4a87-9c92-f97e210612d4" path="/var/lib/kubelet/pods/c5467457-b130-4a87-9c92-f97e210612d4/volumes"
Jul 15 12:56:10.128079 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.128033    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"]
Jul 15 12:56:10.128650 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.128628    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5467457-b130-4a87-9c92-f97e210612d4" containerName="test-curl-7351-140062090032448"
Jul 15 12:56:10.128749 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.128654    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5467457-b130-4a87-9c92-f97e210612d4" containerName="test-curl-7351-140062090032448"
Jul 15 12:56:10.128806 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.128779    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5467457-b130-4a87-9c92-f97e210612d4" containerName="test-curl-7351-140062090032448"
Jul 15 12:56:10.132252 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.132227    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"
Jul 15 12:56:10.134615 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.134585    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:56:10.141749 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.141721    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"]
Jul 15 12:56:10.251647 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.251607    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77165348-dbc6-4483-9645-546fe459e93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-abb283-gs44d\" (UID: \"77165348-dbc6-4483-9645-546fe459e93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"
Jul 15 12:56:10.352384 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.352343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77165348-dbc6-4483-9645-546fe459e93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-abb283-gs44d\" (UID: \"77165348-dbc6-4483-9645-546fe459e93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"
Jul 15 12:56:10.353059 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.353034    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77165348-dbc6-4483-9645-546fe459e93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-abb283-gs44d\" (UID: \"77165348-dbc6-4483-9645-546fe459e93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"
Jul 15 12:56:10.446000 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.445914    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"
Jul 15 12:56:10.591751 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:10.591715    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"]
Jul 15 12:56:10.592441 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:56:10.592409    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77165348_dbc6_4483_9645_546fe459e93e.slice/crio-f75204073602241d278fbaf95ab25b718e40c6ed1d0e54f72055b30dc37877bb WatchSource:0}: Error finding container f75204073602241d278fbaf95ab25b718e40c6ed1d0e54f72055b30dc37877bb: Status 404 returned error can't find the container with id f75204073602241d278fbaf95ab25b718e40c6ed1d0e54f72055b30dc37877bb
Jul 15 12:56:11.310013 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:11.309973    2582 generic.go:356] "Generic (PLEG): container finished" podID="77165348-dbc6-4483-9645-546fe459e93e" containerID="dff7da73f295a30526fb49c38e09244baea701c2abb9f2a13357446d656e20f9" exitCode=0
Jul 15 12:56:11.310445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:11.310065    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d" event={"ID":"77165348-dbc6-4483-9645-546fe459e93e","Type":"ContainerDied","Data":"dff7da73f295a30526fb49c38e09244baea701c2abb9f2a13357446d656e20f9"}
Jul 15 12:56:11.310445 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:11.310150    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d" event={"ID":"77165348-dbc6-4483-9645-546fe459e93e","Type":"ContainerStarted","Data":"f75204073602241d278fbaf95ab25b718e40c6ed1d0e54f72055b30dc37877bb"}
Jul 15 12:56:12.459925 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:12.459899    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"
Jul 15 12:56:12.574490 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:12.574389    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77165348-dbc6-4483-9645-546fe459e93e-maas-api-service-ca\") pod \"77165348-dbc6-4483-9645-546fe459e93e\" (UID: \"77165348-dbc6-4483-9645-546fe459e93e\") "
Jul 15 12:56:12.574851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:12.574816    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77165348-dbc6-4483-9645-546fe459e93e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77165348-dbc6-4483-9645-546fe459e93e" (UID: "77165348-dbc6-4483-9645-546fe459e93e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:56:12.676043 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:12.676001    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77165348-dbc6-4483-9645-546fe459e93e-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:13.323714 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:13.323676    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d"
Jul 15 12:56:13.323872 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:13.323715    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-abb283-gs44d" event={"ID":"77165348-dbc6-4483-9645-546fe459e93e","Type":"ContainerDied","Data":"f75204073602241d278fbaf95ab25b718e40c6ed1d0e54f72055b30dc37877bb"}
Jul 15 12:56:13.323872 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:13.323764    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f75204073602241d278fbaf95ab25b718e40c6ed1d0e54f72055b30dc37877bb"
Jul 15 12:56:20.710579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:20.710541    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"]
Jul 15 12:56:20.711477 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:20.711410    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" podUID="e1b598aa-273c-4899-88d2-906f9ce3f333" containerName="maas-api" containerID="cri-o://c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda" gracePeriod=30
Jul 15 12:56:20.948973 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:20.948914    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" podUID="e1b598aa-273c-4899-88d2-906f9ce3f333" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.88:8443/health\": dial tcp 10.132.0.88:8443: connect: connection refused"
Jul 15 12:56:21.174180 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.174156    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:56:21.255878 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.255794    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls\") pod \"e1b598aa-273c-4899-88d2-906f9ce3f333\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") "
Jul 15 12:56:21.255878 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.255848    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b598aa-273c-4899-88d2-906f9ce3f333-openshift-service-ca\") pod \"e1b598aa-273c-4899-88d2-906f9ce3f333\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") "
Jul 15 12:56:21.256129 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.255931    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfdbw\" (UniqueName: \"kubernetes.io/projected/e1b598aa-273c-4899-88d2-906f9ce3f333-kube-api-access-jfdbw\") pod \"e1b598aa-273c-4899-88d2-906f9ce3f333\" (UID: \"e1b598aa-273c-4899-88d2-906f9ce3f333\") "
Jul 15 12:56:21.256295 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.256271    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1b598aa-273c-4899-88d2-906f9ce3f333-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1b598aa-273c-4899-88d2-906f9ce3f333" (UID: "e1b598aa-273c-4899-88d2-906f9ce3f333"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:56:21.257995 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.257967    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1b598aa-273c-4899-88d2-906f9ce3f333-kube-api-access-jfdbw" (OuterVolumeSpecName: "kube-api-access-jfdbw") pod "e1b598aa-273c-4899-88d2-906f9ce3f333" (UID: "e1b598aa-273c-4899-88d2-906f9ce3f333"). InnerVolumeSpecName "kube-api-access-jfdbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:56:21.258049 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.257972    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1b598aa-273c-4899-88d2-906f9ce3f333" (UID: "e1b598aa-273c-4899-88d2-906f9ce3f333"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:56:21.357458 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.357419    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfdbw\" (UniqueName: \"kubernetes.io/projected/e1b598aa-273c-4899-88d2-906f9ce3f333-kube-api-access-jfdbw\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:21.357458 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.357451    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1b598aa-273c-4899-88d2-906f9ce3f333-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:21.357656 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.357467    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b598aa-273c-4899-88d2-906f9ce3f333-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:21.375690 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.375653    2582 generic.go:356] "Generic (PLEG): container finished" podID="e1b598aa-273c-4899-88d2-906f9ce3f333" containerID="c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda" exitCode=0
Jul 15 12:56:21.375868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.375709    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"
Jul 15 12:56:21.375868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.375738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" event={"ID":"e1b598aa-273c-4899-88d2-906f9ce3f333","Type":"ContainerDied","Data":"c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda"}
Jul 15 12:56:21.375868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.375798    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz" event={"ID":"e1b598aa-273c-4899-88d2-906f9ce3f333","Type":"ContainerDied","Data":"1f94f6875b4b956f92f4364bc42e846042074768c16665159e874fcebd9d623c"}
Jul 15 12:56:21.375868 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.375821    2582 scope.go:117] "RemoveContainer" containerID="c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda"
Jul 15 12:56:21.388625 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.388601    2582 scope.go:117] "RemoveContainer" containerID="c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda"
Jul 15 12:56:21.388973 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:56:21.388955    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda\": container with ID starting with c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda not found: ID does not exist" containerID="c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda"
Jul 15 12:56:21.389051 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.388981    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda"} err="failed to get container status \"c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda\": rpc error: code = NotFound desc = could not find container \"c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda\": container with ID starting with c96d3fd6598ab937d5dd53e7141c3b7b98df92f9fe629036ec9401a85cf69eda not found: ID does not exist"
Jul 15 12:56:21.402133 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.402104    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"]
Jul 15 12:56:21.410268 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:21.410247    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-abb283-54c98b76b5-xgflz"]
Jul 15 12:56:22.064203 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:22.064171    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1b598aa-273c-4899-88d2-906f9ce3f333" path="/var/lib/kubelet/pods/e1b598aa-273c-4899-88d2-906f9ce3f333/volumes"
Jul 15 12:56:36.257853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.257816    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"]
Jul 15 12:56:36.258275 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.258261    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1b598aa-273c-4899-88d2-906f9ce3f333" containerName="maas-api"
Jul 15 12:56:36.258323 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.258277    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1b598aa-273c-4899-88d2-906f9ce3f333" containerName="maas-api"
Jul 15 12:56:36.258323 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.258296    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77165348-dbc6-4483-9645-546fe459e93e" containerName="revoke-keys"
Jul 15 12:56:36.258323 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.258302    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77165348-dbc6-4483-9645-546fe459e93e" containerName="revoke-keys"
Jul 15 12:56:36.258432 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.258369    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77165348-dbc6-4483-9645-546fe459e93e" containerName="revoke-keys"
Jul 15 12:56:36.258432 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.258382    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1b598aa-273c-4899-88d2-906f9ce3f333" containerName="maas-api"
Jul 15 12:56:36.261759 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.261741    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"
Jul 15 12:56:36.264201 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.264176    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hf8ls\""
Jul 15 12:56:36.270810 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.270782    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"]
Jul 15 12:56:36.289553 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.289524    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bbf155-94e5-4df2-9b7d-7a6afeb09000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j\" (UID: \"d1bbf155-94e5-4df2-9b7d-7a6afeb09000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"
Jul 15 12:56:36.389994 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.389954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bbf155-94e5-4df2-9b7d-7a6afeb09000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j\" (UID: \"d1bbf155-94e5-4df2-9b7d-7a6afeb09000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"
Jul 15 12:56:36.390591 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.390568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bbf155-94e5-4df2-9b7d-7a6afeb09000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j\" (UID: \"d1bbf155-94e5-4df2-9b7d-7a6afeb09000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"
Jul 15 12:56:36.573657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.573568    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"
Jul 15 12:56:36.715233 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:56:36.715198    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1bbf155_94e5_4df2_9b7d_7a6afeb09000.slice/crio-73fafb190c3ca5f9d3a12dac1544de53522d17b563c3d1a32e6b82d4678d45d8 WatchSource:0}: Error finding container 73fafb190c3ca5f9d3a12dac1544de53522d17b563c3d1a32e6b82d4678d45d8: Status 404 returned error can't find the container with id 73fafb190c3ca5f9d3a12dac1544de53522d17b563c3d1a32e6b82d4678d45d8
Jul 15 12:56:36.715949 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:36.715915    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"]
Jul 15 12:56:37.479837 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:37.479809    2582 generic.go:356] "Generic (PLEG): container finished" podID="d1bbf155-94e5-4df2-9b7d-7a6afeb09000" containerID="619d360b5895ce2da85d87f3719a235a4a368e0ef1a81c61c77d386218dfb0b1" exitCode=0
Jul 15 12:56:37.480207 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:37.479867    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j" event={"ID":"d1bbf155-94e5-4df2-9b7d-7a6afeb09000","Type":"ContainerDied","Data":"619d360b5895ce2da85d87f3719a235a4a368e0ef1a81c61c77d386218dfb0b1"}
Jul 15 12:56:37.480207 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:37.479920    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j" event={"ID":"d1bbf155-94e5-4df2-9b7d-7a6afeb09000","Type":"ContainerStarted","Data":"73fafb190c3ca5f9d3a12dac1544de53522d17b563c3d1a32e6b82d4678d45d8"}
Jul 15 12:56:38.625450 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:38.625426    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"
Jul 15 12:56:38.810995 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:38.810898    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bbf155-94e5-4df2-9b7d-7a6afeb09000-maas-api-service-ca\") pod \"d1bbf155-94e5-4df2-9b7d-7a6afeb09000\" (UID: \"d1bbf155-94e5-4df2-9b7d-7a6afeb09000\") "
Jul 15 12:56:38.811323 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:38.811298    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1bbf155-94e5-4df2-9b7d-7a6afeb09000-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1bbf155-94e5-4df2-9b7d-7a6afeb09000" (UID: "d1bbf155-94e5-4df2-9b7d-7a6afeb09000"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:56:38.911579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:38.911546    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1bbf155-94e5-4df2-9b7d-7a6afeb09000-maas-api-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:39.495259 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:39.495230    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j"
Jul 15 12:56:39.495433 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:39.495233    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j" event={"ID":"d1bbf155-94e5-4df2-9b7d-7a6afeb09000","Type":"ContainerDied","Data":"73fafb190c3ca5f9d3a12dac1544de53522d17b563c3d1a32e6b82d4678d45d8"}
Jul 15 12:56:39.495433 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:39.495341    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73fafb190c3ca5f9d3a12dac1544de53522d17b563c3d1a32e6b82d4678d45d8"
Jul 15 12:56:46.804850 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:46.804813    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"]
Jul 15 12:56:46.805456 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:46.805137    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792" podUID="d5dcb796-47e5-409a-bc6a-c7308783a558" containerName="maas-api" containerID="cri-o://7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e" gracePeriod=30
Jul 15 12:56:47.272626 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.272604    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:56:47.373160 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.373062    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls\") pod \"d5dcb796-47e5-409a-bc6a-c7308783a558\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") "
Jul 15 12:56:47.373307 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.373165    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ff2c\" (UniqueName: \"kubernetes.io/projected/d5dcb796-47e5-409a-bc6a-c7308783a558-kube-api-access-8ff2c\") pod \"d5dcb796-47e5-409a-bc6a-c7308783a558\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") "
Jul 15 12:56:47.373307 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.373219    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dcb796-47e5-409a-bc6a-c7308783a558-openshift-service-ca\") pod \"d5dcb796-47e5-409a-bc6a-c7308783a558\" (UID: \"d5dcb796-47e5-409a-bc6a-c7308783a558\") "
Jul 15 12:56:47.373610 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.373580    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5dcb796-47e5-409a-bc6a-c7308783a558-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d5dcb796-47e5-409a-bc6a-c7308783a558" (UID: "d5dcb796-47e5-409a-bc6a-c7308783a558"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:56:47.375145 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.375065    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d5dcb796-47e5-409a-bc6a-c7308783a558" (UID: "d5dcb796-47e5-409a-bc6a-c7308783a558"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:56:47.375240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.375164    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5dcb796-47e5-409a-bc6a-c7308783a558-kube-api-access-8ff2c" (OuterVolumeSpecName: "kube-api-access-8ff2c") pod "d5dcb796-47e5-409a-bc6a-c7308783a558" (UID: "d5dcb796-47e5-409a-bc6a-c7308783a558"). InnerVolumeSpecName "kube-api-access-8ff2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:56:47.474677 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.474638    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dcb796-47e5-409a-bc6a-c7308783a558-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:47.474677 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.474668    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ff2c\" (UniqueName: \"kubernetes.io/projected/d5dcb796-47e5-409a-bc6a-c7308783a558-kube-api-access-8ff2c\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:47.474677 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.474678    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dcb796-47e5-409a-bc6a-c7308783a558-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\""
Jul 15 12:56:47.556657 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.556625    2582 generic.go:356] "Generic (PLEG): container finished" podID="d5dcb796-47e5-409a-bc6a-c7308783a558" containerID="7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e" exitCode=0
Jul 15 12:56:47.556820 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.556669    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"
Jul 15 12:56:47.556820 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.556712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792" event={"ID":"d5dcb796-47e5-409a-bc6a-c7308783a558","Type":"ContainerDied","Data":"7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e"}
Jul 15 12:56:47.556820 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.556767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792" event={"ID":"d5dcb796-47e5-409a-bc6a-c7308783a558","Type":"ContainerDied","Data":"9b6efc7bd6af6efb185cbd143731cbe6bb62a3aaaf542cd40beb483a708b4313"}
Jul 15 12:56:47.556820 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.556792    2582 scope.go:117] "RemoveContainer" containerID="7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e"
Jul 15 12:56:47.569177 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.569158    2582 scope.go:117] "RemoveContainer" containerID="7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e"
Jul 15 12:56:47.569484 ip-10-0-135-186 kubenswrapper[2582]: E0715 12:56:47.569457    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e\": container with ID starting with 7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e not found: ID does not exist" containerID="7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e"
Jul 15 12:56:47.569590 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.569493    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e"} err="failed to get container status \"7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e\": rpc error: code = NotFound desc = could not find container \"7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e\": container with ID starting with 7d22d518aa7f108a708a2899e067802091120ee709e81f73cce1b762694e426e not found: ID does not exist"
Jul 15 12:56:47.583504 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.583476    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"]
Jul 15 12:56:47.587240 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:47.587208    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bdc1bb-855b7d6b8b-l7792"]
Jul 15 12:56:48.063337 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:56:48.063303    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5dcb796-47e5-409a-bc6a-c7308783a558" path="/var/lib/kubelet/pods/d5dcb796-47e5-409a-bc6a-c7308783a558/volumes"
Jul 15 12:57:02.699252 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:02.699216    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68b5dccdcc-pvzbl_2c53bec2-a7a0-49c8-9062-c26dd7aafad0/authorino/0.log"
Jul 15 12:57:08.445362 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:08.445318    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-g24bx_8e8d42b7-45a3-44c8-af57-6598de27bff9/manager/0.log"
Jul 15 12:57:08.559923 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:08.559879    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f7b86fcb4-z7zsn_011f2896-4654-4374-8bab-cc76ec767193/manager/0.log"
Jul 15 12:57:08.676460 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:08.676426    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fkrgw_5a8d121a-028a-40e3-9414-583ef7cf9141/server/0.log"
Jul 15 12:57:08.798937 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:08.798888    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j2d2q_b0385147-5995-4d69-b0bd-da7ed08ca180/manager/0.log"
Jul 15 12:57:09.050329 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:09.050222    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27_9c911986-3fd5-482e-b18c-102c44794762/copy-manifests/0.log"
Jul 15 12:57:09.056957 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:09.056928    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27_9c911986-3fd5-482e-b18c-102c44794762/manager/0.log"
Jul 15 12:57:09.573188 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:09.573152    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5bd77b5849-wcbf2_624f60ca-5e9e-4882-a658-833af66d83be/maas-api/0.log"
Jul 15 12:57:09.689529 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:09.689501    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735310-nxzcf_b21b82ed-0447-485d-a6e1-522b0faa4b99/cleanup/0.log"
Jul 15 12:57:09.816716 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:09.816685    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735325-qmvzx_5ea6d91d-d68b-40fc-9f52-a8052d23cda6/cleanup/0.log"
Jul 15 12:57:09.931674 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:09.931592    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-078f33f2-cmtr4_66cefc82-397d-4bd9-aa22-b3b1f8960d47/revoke-keys/0.log"
Jul 15 12:57:10.042505 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.042467    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-a506c9f6-tt89f_f62a707e-f568-43e6-8add-5e8867a0a8d8/revoke-keys/0.log"
Jul 15 12:57:10.180173 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.180138    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-a77f45-nsmkx_57a148ec-ca8c-404e-abd9-b7d890e2794c/revoke-keys/0.log"
Jul 15 12:57:10.311578 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.311544    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-a2a9e1-smwnn_da4dac31-369d-4a17-9196-c128cb707c45/revoke-keys/0.log"
Jul 15 12:57:10.432533 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.432503    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-88ac4482-jxmvs_daaecb73-d9c3-4b4c-977e-75415ac7c8b8/revoke-keys/0.log"
Jul 15 12:57:10.542679 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.542645    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-fec782-p7n4z_e40ec829-e7de-4325-988a-b3bff5561e10/revoke-keys/0.log"
Jul 15 12:57:10.655823 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.655735    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-93edea-wbdkm_f8cc217a-8069-4a2b-a406-c0a23c8e9ba2/revoke-keys/0.log"
Jul 15 12:57:10.772444 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.772410    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-ba8973be-cw7kk_a1419b3a-6a83-4fc7-a7e4-0eae612f206a/revoke-keys/0.log"
Jul 15 12:57:10.892974 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:10.892942    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-4f1f2a1d-xlcm4_78f7501e-de95-461d-99a3-d47072be4cd5/revoke-keys/0.log"
Jul 15 12:57:11.012289 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:11.012263    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-97fdc9d2-g8vh6_5abc9a85-c08a-4b26-b878-ee995e97322a/revoke-keys/0.log"
Jul 15 12:57:11.135855 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:11.135824    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-abb283-gs44d_77165348-dbc6-4483-9645-546fe459e93e/revoke-keys/0.log"
Jul 15 12:57:11.244187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:11.244111    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-bdc1bb-8726j_d1bbf155-94e5-4df2-9b7d-7a6afeb09000/revoke-keys/0.log"
Jul 15 12:57:11.352698 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:11.352610    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-l9kbp_60c0567b-a26c-42ad-b284-088f95564c76/postgres/0.log"
Jul 15 12:57:12.147457 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.147417    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz_db0aa6ab-0337-44c5-a70e-0340151cf8e2/pull/0.log"
Jul 15 12:57:12.157006 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.156981    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz_db0aa6ab-0337-44c5-a70e-0340151cf8e2/extract/0.log"
Jul 15 12:57:12.163582 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.163548    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz_db0aa6ab-0337-44c5-a70e-0340151cf8e2/util/0.log"
Jul 15 12:57:12.284959 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.284912    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59_65b67f40-3daf-4e0f-a45f-c4cff2c91d9b/util/0.log"
Jul 15 12:57:12.291372 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.291329    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59_65b67f40-3daf-4e0f-a45f-c4cff2c91d9b/pull/0.log"
Jul 15 12:57:12.297522 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.297498    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59_65b67f40-3daf-4e0f-a45f-c4cff2c91d9b/extract/0.log"
Jul 15 12:57:12.427429 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.427339    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2_fb0ca6bc-8dd3-477e-ba10-3eeed095742d/util/0.log"
Jul 15 12:57:12.434242 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.434218    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2_fb0ca6bc-8dd3-477e-ba10-3eeed095742d/pull/0.log"
Jul 15 12:57:12.441004 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.440964    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2_fb0ca6bc-8dd3-477e-ba10-3eeed095742d/extract/0.log"
Jul 15 12:57:12.555013 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.554978    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt_573c3e52-822b-4b48-b2f6-dd3f198f9b8b/util/0.log"
Jul 15 12:57:12.562775 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.562748    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt_573c3e52-822b-4b48-b2f6-dd3f198f9b8b/pull/0.log"
Jul 15 12:57:12.570310 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.570286    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt_573c3e52-822b-4b48-b2f6-dd3f198f9b8b/extract/0.log"
Jul 15 12:57:12.693597 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.693513    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68b5dccdcc-pvzbl_2c53bec2-a7a0-49c8-9062-c26dd7aafad0/authorino/0.log"
Jul 15 12:57:12.929448 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:12.929413    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rdr4n_a6ac67d2-d0d9-4917-b077-5f8ab516e89d/manager/0.log"
Jul 15 12:57:13.043002 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:13.042972    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s2n87_598cede8-3607-497d-b472-43800fa6010e/kuadrant-console-plugin/0.log"
Jul 15 12:57:13.160109 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:13.160058    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6vfqz_216d8708-ab10-494c-ba44-e0f7d30e0875/registry-server/0.log"
Jul 15 12:57:13.292010 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:13.291965    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ttdlv_695dde12-6708-44bd-9d58-80d6d0b671ec/manager/0.log"
Jul 15 12:57:13.544161 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:13.544125    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rllfv_77742c0e-8679-4eaa-83ae-1c0e887b79d9/manager/0.log"
Jul 15 12:57:14.248801 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:14.248768    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c594875d4-q5h6s_738e511b-e0b7-46ef-8f29-56b3e5fb0584/kube-auth-proxy/0.log"
Jul 15 12:57:14.484683 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:14.484642    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ldb5r_dcd2520a-433d-457a-9e73-8f983b7ef53d/payload-pre-processing/0.log"
Jul 15 12:57:14.606083 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:14.605997    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-b27b7_ac44c916-a9bf-42e8-9db6-1276498c9fb3/payload-processing/0.log"
Jul 15 12:57:14.730860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:14.730826    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676fc56cdb-2dvx9_a147a563-6d3a-4034-85b2-d05a3b34913c/router/0.log"
Jul 15 12:57:15.076111 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:15.076063    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5shs7_b0de936b-ebf2-4498-97e7-e1fcc765b091/main/0.log"
Jul 15 12:57:15.187544 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:15.187511    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wj9jv_d720a5ec-1c36-45bb-86e4-a10fb4ebbfd2/main/0.log"
Jul 15 12:57:15.420164 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:15.420033    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg96h7_16a01935-c104-45d6-bece-c23ab671c835/main/0.log"
Jul 15 12:57:15.653104 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:15.653073    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-9wmtp_76c783fd-c346-45c2-8039-6f3dd23514c9/main/0.log"
Jul 15 12:57:24.007196 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:24.007151    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h4nz6_dac464eb-a63f-470b-883d-93d07e830acb/global-pull-secret-syncer/0.log"
Jul 15 12:57:24.132886 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:24.132841    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-49slm_2b5741b7-0068-4f5e-9db2-7c0c18e7579c/konnectivity-agent/0.log"
Jul 15 12:57:24.254579 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:24.254545    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-186.ec2.internal_b45ab77f7adac80ae4f46c80d27c83b4/haproxy/0.log"
Jul 15 12:57:28.771904 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.771871    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz_db0aa6ab-0337-44c5-a70e-0340151cf8e2/extract/0.log"
Jul 15 12:57:28.796914 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.796882    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz_db0aa6ab-0337-44c5-a70e-0340151cf8e2/util/0.log"
Jul 15 12:57:28.825473 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.825447    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592bfpz_db0aa6ab-0337-44c5-a70e-0340151cf8e2/pull/0.log"
Jul 15 12:57:28.861009 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.860979    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59_65b67f40-3daf-4e0f-a45f-c4cff2c91d9b/extract/0.log"
Jul 15 12:57:28.886726 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.886696    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59_65b67f40-3daf-4e0f-a45f-c4cff2c91d9b/util/0.log"
Jul 15 12:57:28.913222 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.913191    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j7p59_65b67f40-3daf-4e0f-a45f-c4cff2c91d9b/pull/0.log"
Jul 15 12:57:28.950744 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.950712    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2_fb0ca6bc-8dd3-477e-ba10-3eeed095742d/extract/0.log"
Jul 15 12:57:28.980601 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:28.980571    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2_fb0ca6bc-8dd3-477e-ba10-3eeed095742d/util/0.log"
Jul 15 12:57:29.007187 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.007154    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7359fk2_fb0ca6bc-8dd3-477e-ba10-3eeed095742d/pull/0.log"
Jul 15 12:57:29.040367 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.040273    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt_573c3e52-822b-4b48-b2f6-dd3f198f9b8b/extract/0.log"
Jul 15 12:57:29.066405 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.066387    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt_573c3e52-822b-4b48-b2f6-dd3f198f9b8b/util/0.log"
Jul 15 12:57:29.092075 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.092053    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ltcvt_573c3e52-822b-4b48-b2f6-dd3f198f9b8b/pull/0.log"
Jul 15 12:57:29.443216 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.443123    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68b5dccdcc-pvzbl_2c53bec2-a7a0-49c8-9062-c26dd7aafad0/authorino/0.log"
Jul 15 12:57:29.517584 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.517554    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rdr4n_a6ac67d2-d0d9-4917-b077-5f8ab516e89d/manager/0.log"
Jul 15 12:57:29.546826 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.546800    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s2n87_598cede8-3607-497d-b472-43800fa6010e/kuadrant-console-plugin/0.log"
Jul 15 12:57:29.591768 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.591733    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6vfqz_216d8708-ab10-494c-ba44-e0f7d30e0875/registry-server/0.log"
Jul 15 12:57:29.682482 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.682453    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ttdlv_695dde12-6708-44bd-9d58-80d6d0b671ec/manager/0.log"
Jul 15 12:57:29.778083 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:29.778028    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rllfv_77742c0e-8679-4eaa-83ae-1c0e887b79d9/manager/0.log"
Jul 15 12:57:31.939395 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:31.939360    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mg5n5_7748a1e0-41c6-46d7-9949-1ebd6c67f073/node-exporter/0.log"
Jul 15 12:57:31.974384 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:31.974358    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mg5n5_7748a1e0-41c6-46d7-9949-1ebd6c67f073/kube-rbac-proxy/0.log"
Jul 15 12:57:32.014860 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:32.014830    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mg5n5_7748a1e0-41c6-46d7-9949-1ebd6c67f073/init-textfile/0.log"
Jul 15 12:57:32.731025 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:32.730996    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-dr7tr_75996646-d932-4bca-ba39-06bf99ac8a46/prometheus-operator-admission-webhook/0.log"
Jul 15 12:57:32.767550 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:32.767521    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f6c7669b5-wdznb_8becae98-30ea-46f9-a94d-9114cdec1469/telemeter-client/0.log"
Jul 15 12:57:32.798941 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:32.798915    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f6c7669b5-wdznb_8becae98-30ea-46f9-a94d-9114cdec1469/reload/0.log"
Jul 15 12:57:32.826491 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:32.826468    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f6c7669b5-wdznb_8becae98-30ea-46f9-a94d-9114cdec1469/kube-rbac-proxy/0.log"
Jul 15 12:57:33.178658 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.178571    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"]
Jul 15 12:57:33.179043 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.178991    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5dcb796-47e5-409a-bc6a-c7308783a558" containerName="maas-api"
Jul 15 12:57:33.179043 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.179004    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5dcb796-47e5-409a-bc6a-c7308783a558" containerName="maas-api"
Jul 15 12:57:33.179043 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.179012    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1bbf155-94e5-4df2-9b7d-7a6afeb09000" containerName="revoke-keys"
Jul 15 12:57:33.179043 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.179018    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1bbf155-94e5-4df2-9b7d-7a6afeb09000" containerName="revoke-keys"
Jul 15 12:57:33.179206 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.179087    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1bbf155-94e5-4df2-9b7d-7a6afeb09000" containerName="revoke-keys"
Jul 15 12:57:33.179206 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.179108    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5dcb796-47e5-409a-bc6a-c7308783a558" containerName="maas-api"
Jul 15 12:57:33.182434 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.182414    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.184942 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.184905    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8qq8t\"/\"openshift-service-ca.crt\""
Jul 15 12:57:33.185065 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.184945    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8qq8t\"/\"default-dockercfg-9f9g5\""
Jul 15 12:57:33.185630 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.185612    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8qq8t\"/\"kube-root-ca.crt\""
Jul 15 12:57:33.199551 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.199514    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"]
Jul 15 12:57:33.293693 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.293652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26sgp\" (UniqueName: \"kubernetes.io/projected/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-kube-api-access-26sgp\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.293693 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.293693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-sys\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.293920 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.293835    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-proc\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.293920 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.293869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-podres\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.293920 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.293894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-lib-modules\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395066 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395013    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-sys\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395119    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-proc\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395139    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-sys\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395150    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-podres\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-lib-modules\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395246    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-proc\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395283 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395258    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-podres\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395510 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26sgp\" (UniqueName: \"kubernetes.io/projected/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-kube-api-access-26sgp\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.395510 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.395355    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-lib-modules\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.406580 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.406543    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26sgp\" (UniqueName: \"kubernetes.io/projected/4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb-kube-api-access-26sgp\") pod \"perf-node-gather-daemonset-86p2g\" (UID: \"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb\") " pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.493929 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.493884    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.642074 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.642041    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"]
Jul 15 12:57:33.643624 ip-10-0-135-186 kubenswrapper[2582]: W0715 12:57:33.643596    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4e073fb9_527e_41bc_bc7b_3f2d9bea4fbb.slice/crio-3ddc3fb9c2264fd247534a1556e7eaf2c7dbeb29cd48421f46512ec968cbec74 WatchSource:0}: Error finding container 3ddc3fb9c2264fd247534a1556e7eaf2c7dbeb29cd48421f46512ec968cbec74: Status 404 returned error can't find the container with id 3ddc3fb9c2264fd247534a1556e7eaf2c7dbeb29cd48421f46512ec968cbec74
Jul 15 12:57:33.855853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.855755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g" event={"ID":"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb","Type":"ContainerStarted","Data":"cd6a6fff8a86e4b9dc553f7987fc7e2c993dc43d3c32a8c57216a4156285df29"}
Jul 15 12:57:33.855853 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.855817    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g" event={"ID":"4e073fb9-527e-41bc-bc7b-3f2d9bea4fbb","Type":"ContainerStarted","Data":"3ddc3fb9c2264fd247534a1556e7eaf2c7dbeb29cd48421f46512ec968cbec74"}
Jul 15 12:57:33.856182 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.855946    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:33.877463 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:33.877396    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g" podStartSLOduration=0.877379305 podStartE2EDuration="877.379305ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:57:33.873696973 +0000 UTC m=+2984.973646312" watchObservedRunningTime="2026-07-15 12:57:33.877379305 +0000 UTC m=+2984.977328643"
Jul 15 12:57:34.190712 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:34.190631    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-j648n_5af6bb98-fdb6-44aa-b77a-ec0027e2b90a/networking-console-plugin/0.log"
Jul 15 12:57:35.205875 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:35.205843    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-c95ck_05775dd6-2844-48cb-a259-99f67edf039a/download-server/0.log"
Jul 15 12:57:36.854922 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:36.854881    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7kp2x_34d1fcc1-504d-4d13-bd09-50803e3981df/dns/0.log"
Jul 15 12:57:36.879285 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:36.879256    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7kp2x_34d1fcc1-504d-4d13-bd09-50803e3981df/kube-rbac-proxy/0.log"
Jul 15 12:57:37.009708 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:37.009677    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7bkzq_728cb636-c45f-42d8-a03b-b5f54a4fd18d/dns-node-resolver/0.log"
Jul 15 12:57:37.732460 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:37.732425    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sp6l2_69dae2e5-759c-4555-9ac7-b6ced7bb8177/node-ca/0.log"
Jul 15 12:57:38.931119 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:38.931070    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c594875d4-q5h6s_738e511b-e0b7-46ef-8f29-56b3e5fb0584/kube-auth-proxy/0.log"
Jul 15 12:57:39.045504 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:39.045462    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ldb5r_dcd2520a-433d-457a-9e73-8f983b7ef53d/payload-pre-processing/0.log"
Jul 15 12:57:39.076881 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:39.076844    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-b27b7_ac44c916-a9bf-42e8-9db6-1276498c9fb3/payload-processing/0.log"
Jul 15 12:57:39.099376 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:39.099347    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676fc56cdb-2dvx9_a147a563-6d3a-4034-85b2-d05a3b34913c/router/0.log"
Jul 15 12:57:39.786244 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:39.786215    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqwzl_fc6aefad-a945-40b1-8746-e45c0084de59/serve-healthcheck-canary/0.log"
Jul 15 12:57:39.872739 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:39.872715    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8qq8t/perf-node-gather-daemonset-86p2g"
Jul 15 12:57:40.539313 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:40.539288    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-prhcw_8d0b2eb9-ed8f-4588-87c4-f3184611afa2/kube-rbac-proxy/0.log"
Jul 15 12:57:40.575235 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:40.575205    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-prhcw_8d0b2eb9-ed8f-4588-87c4-f3184611afa2/exporter/0.log"
Jul 15 12:57:40.600902 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:40.600873    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-prhcw_8d0b2eb9-ed8f-4588-87c4-f3184611afa2/extractor/0.log"
Jul 15 12:57:43.280113 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.280054    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-g24bx_8e8d42b7-45a3-44c8-af57-6598de27bff9/manager/0.log"
Jul 15 12:57:43.379593 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.379560    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f7b86fcb4-z7zsn_011f2896-4654-4374-8bab-cc76ec767193/manager/0.log"
Jul 15 12:57:43.406385 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.406360    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fkrgw_5a8d121a-028a-40e3-9414-583ef7cf9141/server/0.log"
Jul 15 12:57:43.410684 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.410665    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c594875d4-q5h6s_738e511b-e0b7-46ef-8f29-56b3e5fb0584/kube-auth-proxy/0.log"
Jul 15 12:57:43.488210 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.488174    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j2d2q_b0385147-5995-4d69-b0bd-da7ed08ca180/manager/0.log"
Jul 15 12:57:43.502851 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.502819    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ldb5r_dcd2520a-433d-457a-9e73-8f983b7ef53d/payload-pre-processing/0.log"
Jul 15 12:57:43.525144 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.525083    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-b27b7_ac44c916-a9bf-42e8-9db6-1276498c9fb3/payload-processing/0.log"
Jul 15 12:57:43.541723 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.541637    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676fc56cdb-2dvx9_a147a563-6d3a-4034-85b2-d05a3b34913c/router/0.log"
Jul 15 12:57:43.628488 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.628462    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27_9c911986-3fd5-482e-b18c-102c44794762/manager/0.log"
Jul 15 12:57:43.654239 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:43.654213    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd4cf8ff5-f8v27_9c911986-3fd5-482e-b18c-102c44794762/copy-manifests/0.log"
Jul 15 12:57:44.914296 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:44.914264    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c594875d4-q5h6s_738e511b-e0b7-46ef-8f29-56b3e5fb0584/kube-auth-proxy/0.log"
Jul 15 12:57:44.996023 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:44.995992    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-ldb5r_dcd2520a-433d-457a-9e73-8f983b7ef53d/payload-pre-processing/0.log"
Jul 15 12:57:45.016574 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:45.016539    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-b27b7_ac44c916-a9bf-42e8-9db6-1276498c9fb3/payload-processing/0.log"
Jul 15 12:57:45.035131 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:45.035083    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676fc56cdb-2dvx9_a147a563-6d3a-4034-85b2-d05a3b34913c/router/0.log"
Jul 15 12:57:49.683200 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:49.683173    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"
Jul 15 12:57:49.698600 ip-10-0-135-186 kubenswrapper[2582]: I0715 12:57:49.698570    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9sx9r_ec463e22-c92e-4d8a-9f91-a60f8bbc6e24/ovn-acl-logging/0.log"